A map-based system using speech and 3D gestures for pervasive computing

A. Corradini, R. M. Wesson, P. R. Cohen

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

27 Citations (Scopus)


We describe an augmentation of Quickset, a multimodal voice/pen system that allows users to create and control map-based, collaborative, interactive simulations. In this paper, we report on our extension of the graphical pen input mode front stylus/mouse to 3D hand movements. To do this, the map is projected onto a virtual plane in space, specified by the operator before the start of the interactive session. We then use our geometric model to compute the intersection of hand movements with the virtual plane, translating these into map coordinates on the appropriate system. The goal of this research is the creation of a body-centered, multimodal architecture employing both speech and 3D hand gestures, which seamlessly, and unobtrusively supports distributed interaction. The augmented system, built on top of an existing architecture, also provides an improved visualization, management and awareness of a shared understanding. Potential applications of this work include telemedicine, battlefield management and any kind of collaborative decision-making during which users may wish to be mobile.

Original languageEnglish
Title of host publicationProceedings - 4th IEEE International Conference on Multimodal Interfaces, ICMI 2002
PublisherIEEE, Institute of Electrical and Electronics Engineers
Number of pages6
ISBN (Electronic)0769518346, 9780769518343
Publication statusPublished - 1 Jan 2002
Externally publishedYes
EventInternational Conference on Multimodal Interfaces 2002 - Pittsburgh, United States of America
Duration: 14 Oct 200216 Oct 2002
Conference number: 4th
https://ieeexplore.ieee.org/xpl/conhome/8346/proceeding (Proceedings)

Publication series

NameProceedings - 4th IEEE International Conference on Multimodal Interfaces, ICMI 2002


ConferenceInternational Conference on Multimodal Interfaces 2002
Abbreviated titleICMI 2002
Country/TerritoryUnited States of America
Internet address

Cite this