An Agent-based System for
Capturing and Indexing Software Design Meetings
|
|
|
Tracy Hammond, Design Rationale Group |
|
Krzysztof Gajos, Intelligent Room |
|
Randall Davis, Design Rationale Group |
|
Howard Shrobe, Intelligent Room |
Software Design Meetings
Capturing Design History
|
|
|
|
Videotape the meeting |
|
Videotape the white board |
|
|
|
Question: |
|
How do you index videotape? |
Solution: Tahuti, Sketch
Recognition of Design Diagrams
|
|
|
UML
Diagrams + Agents |
|
Natural: Shape Based |
|
Projected Screen / Digital Pen |
|
Agent Arch. – Many Computers, Devices |
Tahuti: UML Sketch
Recognition
|
|
|
Sketch Naturally |
|
Natural Editing Capabilities |
|
Code Generation Facilities |
|
Event Indexing |
|
|
Tahuti: UML Sketch
Recognition
|
|
|
Sketch Naturally |
|
Natural Editing Capabilities |
|
Code Generation Facilities |
|
Event Indexing |
|
|
Tahuti: UML Sketch
Recognition
|
|
|
Sketch Naturally |
|
Natural Editing Capabilities |
|
Code Generation Facilities |
|
Event Indexing |
|
|
Recognizable Shapes
|
|
|
|
UML Class Diagram |
|
General Class |
|
Interface Class |
|
Interface Association |
|
Dependency Association |
|
Inheritance Association |
|
Aggregation Association |
|
Intelligent Room Agents |
|
Agent Class |
|
Speech Grammar |
|
|
Sketched Diagram
Interpreted Diagram
Agent Abstraction
Relies On Abstraction
Speech Grammar
Event Indexing
|
|
|
|
Events |
|
Addition, Deletion, Movement of General
Class, Interface Class, Agent Class, Sketch Grammar, Association,
Unrecognized Stroke |
|
Text update |
|
Undo, Redo |
|
Use to Index Videotape |
|
What was the meeting discussion when
this class was added? |
Auto-summarization of
Design History by Event Ranking
|
|
|
Final Design: 12 |
|
User Marked Significant Event: 11 |
|
Creation of an Existing Agent: 10 |
|
Creation of an Existing General or
Interface Class: 9 |
|
Creation of an Existing Speech Grammar:
8 |
|
Creation of an Existing Association: 7 |
|
Creation of an Existing Unrecognized
Stroke: 6 |
|
Text Update: 5 |
|
Movement: 4 |
|
Creation of Deleted Object: 3 |
|
Deletion of an Object: 2 |
|
Undo/Redo: 1 |
Auto-summarization of
Design History by Event Ranking
Design History
Tahuti Architecture
Metaglue Agent
Architecture
|
|
|
Support for synchronous and
asynchronous communication among distributed agents |
|
Mechanisms for resource discovery and
management |
|
Robust recovery mechanisms for failed
components |
|
Built-in persistent storage |
|
Support for multimodal interactions
through speech, gesture, and graphical user interfaces. |
Impact and Current Use
|
|
|
|
UserStudy |
|
Users Prefer Tahuti for drawing and
editing UML diagrams over MS Paint or Rationale Rose |
|
Deployment |
|
Used to teach 65 students Object
Oriented Programming at Columbia University |
|
Used in the MIT AI Lab Intelligent Room
for use in software design meetings.
Currently extending to add speech interface. |
Related Work
|
|
|
|
UML Sketch Recognition |
|
Hse – Wizard of Oz – prefer to mouse
and palette |
|
Ideogramic UML, graffiti-like
implementation |
|
Queen’s Unversity, based on stroke
length |
|
UML for Designing Agent Based Systems |
|
Odell, Parunak and Bauer |
|
Bergenti and Poggi |
|
Indexing Audio-Visual Material |
|
Brunelli, Mich, and Modena |
|
Sketch Recognition to Locate Video
Moment |
|
Kato, Kurita, Otsu, and Hirata |
|
Cho and Yoo |
|
Jacobs, Finkelstein, and Salesin |
Future Work
|
|
|
|
Expansion of system |
|
Multiplicity Relationships |
|
More Symbols |
|
Deeper layers of software design |
|
Sequence Diagrams |
|
Flow Charts |
|
Generate More Code |
|
Code Editor |
Contributions
|
|
|
|
Agent-based system for capturing and
indexing software design meetings |
|
Natural Sketch Recognition |
|
Supports Multi-Agent Software Design |
|
Indexes Design Meeting Videos |