The role of natural language in a multimodal interface

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

75 Citations (Scopus)

Abstract

Although graphics and direct manipulation are effective interface technologies for some classes of problems, they are limited in many ways. In particular, they provide little support for identifying objects not on the screen, for specifying temporal relations, for identifying and operating on large sets and subsets of entities, and for using the context of interaction. On the other hand, these are precisely strengths of natural language. This paper presents an interface that blends natural language processing and direct manipulation technologies, using each for their characteristic advantages. Specifically, the paper shows how to use natural language to describe objects and temporal relations, and how to use direct manipulation for overcoming hard natural language p roblems involving the establishment and use of context and pronominal reference. This work has been implemented in SRI's Shoptalk system, a prototype information and decision-support system for manufacturing.

Original languageEnglish
Title of host publicationProceedings of the 5th Annual ACM Symposium on User Interface Software and Technology, UIST 1992
PublisherAssociation for Computing Machinery (ACM)
Pages143-149
Number of pages7
ISBN (Electronic)0897915496, 9780897915496
DOIs
Publication statusPublished - 1 Dec 1992
Externally publishedYes
EventACM Symposium on User Interface Software and Technology 1992 - Monteray, United States of America
Duration: 15 Nov 199218 Nov 1992
Conference number: 5th
https://dl.acm.org/doi/proceedings/10.1145/142621

Publication series

NameProceedings of the 5th Annual ACM Symposium on User Interface Software and Technology, UIST 1992

Conference

ConferenceACM Symposium on User Interface Software and Technology 1992
Abbreviated titleUIST 1992
CountryUnited States of America
CityMonteray
Period15/11/9218/11/92
Internet address

Cite this