Demo: Collaborative multimodal photo annotation over digital paper

Paulo Barthelmess, Edward Kaiser, Xiao Huang, David McGee, Philip Cohen

Research output: Chapter in Book/Report/Conference proceedingConference PaperOtherpeer-review

3 Citations (Scopus)


The availability of metadata annotations over media content such as photos is known to enhance retrieval and organization, particularly for large data sets. The greatest challenge for obtaining annotations remains getting users to perform the large amount of tedious manual work that is required. In this demo we show a system for semi-automated labeling based on extraction of metadata from naturally occurring conversations of groups of people discussing pictures among themselves. The system supports a variety of collaborative label elicitation scenarios mixing co-located and distributed participants, operating primarily via speech, handwriting and sketching over tangible digital paper photo printouts. We demonstrate the real-time capabilities of the system by providing hands-on annotation experience for conference participants. Demo annotations are performed over public domain pictures portraying mainstream themes (e.g. from famous movies).

Original languageEnglish
Title of host publicationICMI'06
Subtitle of host publication8th International Conference on Multimodal Interfaces, Conference Proceedings
PublisherAssociation for Computing Machinery (ACM)
Number of pages2
ISBN (Print)159593541X, 9781595935410
Publication statusPublished - 1 Dec 2006
Externally publishedYes
EventInternational Conference on Multimodal Interfaces 2006 - Banff, Canada
Duration: 2 Nov 20064 Nov 2006
Conference number: 8th (Proceedings)


ConferenceInternational Conference on Multimodal Interfaces 2006
Abbreviated titleICMI 2006
Internet address


  • Collaborative interaction
  • Demo
  • Intelligent interfaces
  • Multimodal processing
  • Photo annotation

Cite this