Auditory User Assistance Systems
Aidan Kehoe, Flaithri Neff, Ian Pitt
Speech Technology
- Speech technology is widely available...
- Speech recognition/synthesis modules are available in Windows and other operating systems
- All modern web-browsers are speech-enabled (natively or via plug-ins)
- ...but little used.
We believe speech is under-used because:
- Developers don't recognise its potential
- There are few design aids available (models, guidelines, metrics, etc.)
We are developing:
- A model of multi-modal (auditory/visual) interaction
- Test and demonstration systems using speech and graphics
Possible Applications
- User Assistance (help) systems:
- User receives help information via speech
- User navigates through help topics using voice commands
- Application remains fully visible on screen (no help pane)
- Hands remain free to operate application
- Control of system/application parameters
- Vary mouse gain during drag/drop operations
- Change view, mode, etc.
Test System - A Computer Game
- Game audio played over loudspeakers
- Help provided using speech commands
and voice feedback via headset - Control of mouse-gain and other parameters
- Speech commands via headset
- Current settings displayed on auxilliary LCD
(attached to external keyboard)
Publications
Kehoe, A., Neff, F. & Pitt, I.J. (2007), Modifications to a Speech-Enabled User Assistance System Based on Pilot Study Results, accepted for presentation at the ACM International Conference on Design of Communication (SIGDOC), El Paso, Texas, USA, October 22-24th 2007
Kehoe, A. & Pitt, I.J. (2007), Transforming DITA Topics for Speech Synthesis Output, in proceedings of Hypertext 2007, Manchester, UK, 10-12th September 2007
Neff, F., Kehoe, A. & Pitt, I.J. (2007), User Modeling to Support the Development of an Auditory Help System, in proceedings of the International Conference on Text, Speech and Dialogue (TSD), Pilsen, Czech Republic, 3-7th September 2007
Kehoe, A. & Pitt, I.J. (2007), Use of Voice Input to Enhance Cursor Control in Mainstream Gaming Applications, in proceedings of the ACM CHI 2007 workshop "Striking a C[h]ord: Vocal Interaction in Assistive Technologies, Games, and More", San Jose, California, 28 April-3 May 2007
Kehoe, A., Neff, F. and Pitt, I.J. (2007), Extending traditional user assistance systems to support an auditory interface, Artificial Intelligence and Applications (AIA 2007), Innsbruck, Austria, 12-14 February 2007
Kehoe, A., Neff, F. and Pitt, I.J. (2006), Using Speech Technology to provide Context-Specific Help in Games, Computer Games (CGames 2006), Dublin, Ireland, 22-24th November 2006
Kehoe, A., Neff, F. and Pitt, I.J. (2006), Design and Implementation of an Auditory Help System for Computer Games, Game Design and Technology Workshop, Liverpool, November 2006
Kehoe, A. and Pitt, I.J. (2006), Designing Help Topics for use with Text-To-Speech, International Conference on Design of Communication, Myrtle Beach, SC, October 2006.
Kehoe, A. and Pitt, I.J. (2006), Using Speech Technology to Provide Help in Games, Proceedings of the Fun 'n Games Conference, Preston, England, June 2006