Prototyping novel collaborative multimodal systems: simulation, data collection and analysis tools for the next decade

Alexander M. Arthur, Rebecca Lunsford, Matt Wesson, Sharon Oviatt

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

15 Citations (Scopus)


To support research and development of next-generation multimodal interfaces for complex collaborative tasks, a comprehensive new infrastructure has been created for collecting and analyzing time-synchronized audio, video, and pen-based data during multi-party meetings. This infrastructure needs to be unobtrusive and to collect rich data involving multiple information sources of high temporal fidelity to allow the collection and annotation of simulation-driven studies of natural human-human-computer interactions. Furthermore, it must be flexibly extensible to facilitate exploratory research. This paper describes both the infrastructure put in place to record, encode, playback and annotate the meeting-related media data, and also the simulation environment used to prototype novel system concepts.

Original languageEnglish
Title of host publicationICMI '06 - Proceedings of the 8th international conference on Multimodal interfaces
Place of PublicationNew York NY USA
PublisherAssociation for Computing Machinery (ACM)
Number of pages8
ISBN (Print)159593541X, 9781595935410
Publication statusPublished - 2006
Externally publishedYes
EventInternational Conference on Multimodal Interfaces 2006 - Banff, Canada
Duration: 2 Nov 20064 Nov 2006
Conference number: 8th (Proceedings)


ConferenceInternational Conference on Multimodal Interfaces 2006
Abbreviated titleICMI 2006
Internet address


  • Annotation tools
  • Data collection infrastructure
  • Meeting
  • Multi-party
  • Multimodal interfaces
  • Simulation studies
  • Synchronized media

Cite this