Just do what I tell you: the limited impact of instructions on multimodal integration patterns

Sharon Oviatt, Rachel Coulston, Rebecca Lunsford

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

2 Citations (Scopus)


Large individual differences have been documented among users in their multimodal integration patterns, which suggest that new user-adaptive approaches to multimodal fusion may be opportune. Before pursuing such an approach, this study explores whether people can be successfully encouraged to switch their multimodal integration pattern to one that is easier to process through the use of explicit instructions. Longitudinal data were collected from young and elderly adults as they used speech and pen input with a simulated map system. Results revealed that only 37% of users switched their integration pattern and maintained it, whereas another 19% never switched their natural pattern and 31% switched but then reverted during a follow-up session. In addition, significant destabilization of elderly users' integration pattern was one "cost" of attempting to instruct a change in pattern. This research underscores the need for user-centered design in future multimodal system development, especially for vulnerable users such as the elderly.

Original languageEnglish
Title of host publication10th International Conference on User Modeling 2005, Edinburgh, Scotland, United Kingdom, 24 July 2005 through 29 July 2005
Place of PublicationCham Switzerland
Number of pages10
ISBN (Electronic)3540278850
ISBN (Print)9783540278856
Publication statusPublished - 2005
Externally publishedYes
Event10th International Conference on User Modeling 2005, UM 2005 - Edinburgh, Scotland, United Kingdom
Duration: 24 Jul 200529 Jul 2005

Publication series

NameLecture Notes in Computer Science
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


Conference10th International Conference on User Modeling 2005, UM 2005
CountryUnited Kingdom
CityEdinburgh, Scotland

Cite this