OpenSense: a platform for multimodal data acquisition and behavior perception

Kalin Stefanov, Baiyu Huang, Zongjian Li, Mohammad Soleymani

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

2 Citations (Scopus)


Automatic multimodal acquisition and understanding of social signals is an essential building block for natural and effective human-machine collaboration and communication. This paper introduces OpenSense, a platform for real-time multimodal acquisition and recognition of social signals. OpenSense enables precisely synchronized and coordinated acquisition and processing of human behavioral signals. Powered by the Microsoft's Platform for Situated Intelligence, OpenSense supports a range of sensor devices and machine learning tools and encourages developers to add new components to the system through straightforward mechanisms for component integration. This platform also offers an intuitive graphical user interface to build application pipelines from existing components. OpenSense is freely available for academic research.

Original languageEnglish
Title of host publicationProceedings of the 2020 International Conference on Multimodal Interaction
EditorsNadia Berthouze, Mohamed Chetouani, Mikio Nakano
Place of PublicationNew York NY USA
PublisherAssociation for Computing Machinery (ACM)
Number of pages5
ISBN (Electronic)9781450375818
Publication statusPublished - 2020
Externally publishedYes
EventInternational Conference on Multimodal Interaction 2020 - Virtual , Netherlands
Duration: 25 Oct 202029 Oct 2020
Conference number: 22nd (Website) (Proceedings)


ConferenceInternational Conference on Multimodal Interaction 2020
Abbreviated titleICMI 2020
Internet address


  • behavior
  • multimodal
  • open source
  • perception
  • platform

Cite this