Abstract
We describe an approach to 3D multimodal interaction in immersive augmented and virtual reality environments that accounts for the uncertain nature of the information sources. The resulting multimodal system fuses symbolic and statistical information from a set of 3D gesture, spoken language, and referential agents. The referential agents employ visible or invisible volumes that can be attached to 3D trackers in the environment, and which use a time-stamped history of the objects that intersect them to derive statistics for ranking potential referents. We discuss the means by which the system supports mutual disambiguation of these modalities and information sources, and show through a user study how mutual disambiguation accounts for over 45% of the successful 3D multimodal interpretations. An accompanying video demonstrates the system in action.
Original language | English |
---|---|
Title of host publication | ICMI'03 |
Subtitle of host publication | Fifth International Conference on Multimodal Interfaces |
Pages | 12-19 |
Number of pages | 8 |
Publication status | Published - 1 Dec 2003 |
Externally published | Yes |
Event | International Conference on Multimodal Interfaces 2003 - Vancouver, Canada Duration: 5 Nov 2003 → 7 Nov 2003 Conference number: 5th https://dl.acm.org/doi/proceedings/10.1145/958432 (Proceedings) |
Publication series
Name | ICMI'03: Fifth International Conference on Multimodal Interfaces |
---|
Conference
Conference | International Conference on Multimodal Interfaces 2003 |
---|---|
Abbreviated title | ICMI 2003 |
Country | Canada |
City | Vancouver |
Period | 5/11/03 → 7/11/03 |
Internet address |
|
Keywords
- Augmented/virtual reality
- Evaluation
- Multimodal interaction