Multimodal Interaction with Computers

P. R. Cohen, S. L. Oviatt

Research output: Chapter in Book/Report/Conference proceedingChapter (Book)Researchpeer-review

1 Citation (Scopus)


Multimodal interaction involves a person's use of more than one of their natural communication modes, such as spoken language, gesture, sketch, writing, eye gaze, body posture, and so forth, to communicate with a partner. For a computer to respond appropriately, it will need to derive coherent communicative intent from this combined set of inputs. This article summarizes the advantages of multimodal interaction, progress in our understanding of humans' multimodal performance, the state of the art in multimodal system processing, and challenges and opportunities for future science and technology of multimodal interaction. This article does not cover multimodal presentation planning and output.

Original languageEnglish
Title of host publicationEncyclopedia of Language & Linguistics
PublisherAcademic Press
Number of pages11
ISBN (Print)9780080448541
Publication statusPublished - 1 Dec 2006
Externally publishedYes

Cite this