Voice-Driven Drawing Tool

Software	Object-oriented language with good GUI, speech recognition software
Covers	Interface design, software development, possibly XML
Skills Required	Programming, ideally XML
Challenge	Conceptual ?? Technical ??? Programming ????

Brief Description

The aim of the project is to develop a concept demonstrator prototype of a computer drawing tool where the user selects options and modes, and puts in text input, using voice commands. This should make using the drawing tool much more fluent at the expense of making it much noiser.

The project will involve creating an interface that accepts voice commands. This is probably best done using the API to the Microsoft speech recognition software but other possibilities exist. It will need to recognise a set of commands and possess the ability to accept unrestricted voice input of text. The interface will also need the ability to create and position a range of shapes, and if possible, allow the user to move and resize them. Ideally it should automatically redraw connecting lines when objects are moved. (Whether something is a connecting line or just a line or a freeform mark is something that could be controlled by voice input of an appropriate command.) The system should enable the user to save and reload drawings; this is probably best done by finding and using an existing document format.

Variants

It might be a good idea to focus on a narrow range of types of drawing or diagram, or on a particular application. Possibilities include the following...

Freeform drawings, where a user can create standard shapes and freeform lines, and choose types of marks, colours, etc, by voice commands.
Mind maps, where users create nodes with labels and lines linking them, where the ability to talk and draw at the same time to smooth the interaction with the interface would enable much more fluent idea exploration. Desirable features might be having different kinds of connection lines, and showing or hiding particular kinds; adding additional text to a node and showing or hiding it on command; and being able to swap betweeen only showing the tree structure formed by the connections branching out from a particular root node, and showing all the connections in a real mind map.
UML diagrams, such as activity diagrams and state machine diagrams.
Networks of causal relationships.

Back to