Synergistic use of direct manipulation and natural language

Philip R. Cohen, Mary Dalrymple, Douglas B. Moran, Fernando C N Pereira, Joseph W. Sullivan, Robert A. Gargan, Jon L. Schlossberg, Sherman W. Tyler

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

73 Citations (Scopus)


This paper shows how the integration of natural language with direct manipulation produces a multimodal interface that overcomes limitations of these techniques when used separately. Natural language helps direct manipulation in being able to specify objects and actions by description, while direct manipulation enables users to learn which objects and actions are available in the system. Furthermore, graphical rendering and manipulation of context provides a partial solution to difficult problems of natural language anaphora.

Original languageEnglish
Title of host publicationProceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI 1989
PublisherAssociation for Computing Machinery (ACM)
Number of pages7
ISBN (Print)0897913019, 9780897913010
Publication statusPublished - 1 Jan 1989
Externally publishedYes
EventInternational Conference on Human Factors in Computing Systems 1989 - Austin, United States of America
Duration: 30 Apr 19894 Jun 1989
Conference number: 7th

Publication series

NameConference on Human Factors in Computing Systems - Proceedings


ConferenceInternational Conference on Human Factors in Computing Systems 1989
Abbreviated titleCHI 1989
Country/TerritoryUnited States of America


  • Direct manipulation
  • Natural language
  • User interfaces

Cite this