OpenSense: a platform for multimodal data acquisition and behavior perception

Kalin Stefanov, Baiyu Huang, Zongjian Li, Mohammad Soleymani

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

Abstract

Automatic multimodal acquisition and understanding of social signals is an essential building block for natural and effective human-machine collaboration and communication. This paper introduces OpenSense, a platform for real-time multimodal acquisition and recognition of social signals. OpenSense enables precisely synchronized and coordinated acquisition and processing of human behavioral signals. Powered by the Microsoft's Platform for Situated Intelligence, OpenSense supports a range of sensor devices and machine learning tools and encourages developers to add new components to the system through straightforward mechanisms for component integration. This platform also offers an intuitive graphical user interface to build application pipelines from existing components. OpenSense is freely available for academic research.

Original languageEnglish
Title of host publicationProceedings of the 2020 International Conference on Multimodal Interaction
EditorsNadia Berthouze, Mohamed Chetouani, Mikio Nakano
Place of PublicationNew York NY USA
PublisherAssociation for Computing Machinery (ACM)
Pages660-664
Number of pages5
ISBN (Electronic)9781450375818
DOIs
Publication statusPublished - 2020
Externally publishedYes
EventInternational Conference on Multimodal Interaction 2020 - Virtual , Netherlands
Duration: 25 Oct 202029 Oct 2020
Conference number: 22nd
https://icmi.acm.org/2020/index.php?id=cfp (Website)
https://dl.acm.org/doi/proceedings/10.1145/3382507 (Proceedings)

Conference

ConferenceInternational Conference on Multimodal Interaction 2020
Abbreviated titleICMI 2020
CountryNetherlands
CityVirtual
Period25/10/2029/10/20
Internet address

Keywords

  • behavior
  • multimodal
  • open source
  • perception
  • platform

Cite this