Multimodal interfaces: combining interfaces to accomplish a single task

Paulo Barthelmess, Sharon Oviatt

Research output: Chapter in Book/Report/Conference proceedingChapter (Book)Researchpeer-review

4 Citations (Scopus)

Abstract

An essential distinguishing capability of multimodal systems is the use these systems make of two or more natural input modalities such as speech, handwriting, gestures, facial expressions, and other body movements. Using such systems, users deal with a crisis management situation using speech and pen input over a map. An emergency response route can be established by sketching a line on a map along the desired route, using a digital pen or a Tablet PC stylus, while speaking "Create emergency route here." Multimodal interfaces take many forms and have many aspects. Small portable devices host systems, such as on a cell phone that is taken to the field or on a tablet computer used within offices or in cafeterias. Groups of people may interact multimodally via large interactive boards or sheets of digital paper. The common aspect in all these cases is the interface support for interaction via natural modes of communication involving combinations, for example, of speech, pen input, gestures, gaze, or other modalities. The intrinsic advantage of multimodal systems is that they allow users to convey their intentions in a more expressive way, better matched to the way they naturally communicate. A well-designed multimodal system gives users the freedom to choose the modality that they feel best matches the requirements of the task at hand.

Original languageEnglish
Title of host publicationHCI Beyond the GUI
Subtitle of host publicationDesign for Haptic, Speech, Olfactory, and Other Nontraditional Interfaces
PublisherAcademic Press
Pages391-444
Number of pages54
ISBN (Print)9780123740175
DOIs
Publication statusPublished - 2008
Externally publishedYes

Cite this

Barthelmess, P., & Oviatt, S. (2008). Multimodal interfaces: combining interfaces to accomplish a single task. In HCI Beyond the GUI: Design for Haptic, Speech, Olfactory, and Other Nontraditional Interfaces (pp. 391-444). Academic Press. https://doi.org/10.1016/B978-0-12-374017-5.00012-2