OpenSense: a platform for multimodal data acquisition and behavior perception

Kalin Stefanov; Baiyu Huang; Zongjian Li; Mohammad Soleymani

doi:10.1145/3382507.3418832

OpenSense: a platform for multimodal data acquisition and behavior perception

Kalin Stefanov, Baiyu Huang, Zongjian Li, Mohammad Soleymani

Research output: Chapter in Book/Report/Conference proceeding › Conference Paper › Research › peer-review

Abstract

Automatic multimodal acquisition and understanding of social signals is an essential building block for natural and effective human-machine collaboration and communication. This paper introduces OpenSense, a platform for real-time multimodal acquisition and recognition of social signals. OpenSense enables precisely synchronized and coordinated acquisition and processing of human behavioral signals. Powered by the Microsoft's Platform for Situated Intelligence, OpenSense supports a range of sensor devices and machine learning tools and encourages developers to add new components to the system through straightforward mechanisms for component integration. This platform also offers an intuitive graphical user interface to build application pipelines from existing components. OpenSense is freely available for academic research.

Original language	English
Title of host publication	Proceedings of the 2020 International Conference on Multimodal Interaction
Editors	Nadia Berthouze, Mohamed Chetouani, Mikio Nakano
Place of Publication	New York NY USA
Publisher	Association for Computing Machinery (ACM)
Pages	660-664
Number of pages	5
ISBN (Electronic)	9781450375818
DOIs	https://doi.org/10.1145/3382507.3418832
Publication status	Published - 2020
Externally published	Yes
Event	International Conference on Multimodal Interaction 2020 - Virtual , Netherlands Duration: 25 Oct 2020 → 29 Oct 2020 Conference number: 22nd https://icmi.acm.org/2020/index.php?id=cfp (Website) https://dl.acm.org/doi/proceedings/10.1145/3382507 (Proceedings)

Conference

Conference	International Conference on Multimodal Interaction 2020
Abbreviated title	ICMI 2020
Country	Netherlands
City	Virtual
Period	25/10/20 → 29/10/20
Internet address	https://icmi.acm.org/2020/index.php?id=cfp (Website) https://dl.acm.org/doi/proceedings/10.1145/3382507 (Proceedings)

Keywords

behavior
multimodal
open source
perception
platform

Access to Document

10.1145/3382507.3418832

Link to publication in Scopus

Cite this

@inproceedings{c6aab8aa177d45e0b95acf5a72a463b4,

title = "OpenSense: a platform for multimodal data acquisition and behavior perception",

abstract = "Automatic multimodal acquisition and understanding of social signals is an essential building block for natural and effective human-machine collaboration and communication. This paper introduces OpenSense, a platform for real-time multimodal acquisition and recognition of social signals. OpenSense enables precisely synchronized and coordinated acquisition and processing of human behavioral signals. Powered by the Microsoft's Platform for Situated Intelligence, OpenSense supports a range of sensor devices and machine learning tools and encourages developers to add new components to the system through straightforward mechanisms for component integration. This platform also offers an intuitive graphical user interface to build application pipelines from existing components. OpenSense is freely available for academic research. ",

keywords = "behavior, multimodal, open source, perception, platform",

author = "Kalin Stefanov and Baiyu Huang and Zongjian Li and Mohammad Soleymani",

year = "2020",

doi = "10.1145/3382507.3418832",

language = "English",

pages = "660--664",

editor = "Berthouze, {Nadia } and Chetouani, {Mohamed } and Nakano, {Mikio }",

booktitle = "Proceedings of the 2020 International Conference on Multimodal Interaction",

publisher = "Association for Computing Machinery (ACM)",

address = "United States of America",

note = "International Conference on Multimodal Interaction 2020, ICMI 2020 ; Conference date: 25-10-2020 Through 29-10-2020",

url = "https://icmi.acm.org/2020/index.php?id=cfp, https://dl.acm.org/doi/proceedings/10.1145/3382507",

}

Stefanov, K, Huang, B, Li, Z & Soleymani, M 2020, OpenSense: a platform for multimodal data acquisition and behavior perception. in N Berthouze, M Chetouani & M Nakano (eds), Proceedings of the 2020 International Conference on Multimodal Interaction. Association for Computing Machinery (ACM), New York NY USA, pp. 660-664, International Conference on Multimodal Interaction 2020, Virtual , Netherlands, 25/10/20. https://doi.org/10.1145/3382507.3418832

OpenSense : a platform for multimodal data acquisition and behavior perception. / Stefanov, Kalin; Huang, Baiyu; Li, Zongjian; Soleymani, Mohammad.

Proceedings of the 2020 International Conference on Multimodal Interaction. ed. / Nadia Berthouze; Mohamed Chetouani; Mikio Nakano. New York NY USA : Association for Computing Machinery (ACM), 2020. p. 660-664.

Research output: Chapter in Book/Report/Conference proceeding › Conference Paper › Research › peer-review

TY - GEN

T1 - OpenSense

T2 - International Conference on Multimodal Interaction 2020

AU - Stefanov, Kalin

AU - Huang, Baiyu

AU - Li, Zongjian

AU - Soleymani, Mohammad

N1 - Conference code: 22nd

PY - 2020

Y1 - 2020

N2 - Automatic multimodal acquisition and understanding of social signals is an essential building block for natural and effective human-machine collaboration and communication. This paper introduces OpenSense, a platform for real-time multimodal acquisition and recognition of social signals. OpenSense enables precisely synchronized and coordinated acquisition and processing of human behavioral signals. Powered by the Microsoft's Platform for Situated Intelligence, OpenSense supports a range of sensor devices and machine learning tools and encourages developers to add new components to the system through straightforward mechanisms for component integration. This platform also offers an intuitive graphical user interface to build application pipelines from existing components. OpenSense is freely available for academic research.

AB - Automatic multimodal acquisition and understanding of social signals is an essential building block for natural and effective human-machine collaboration and communication. This paper introduces OpenSense, a platform for real-time multimodal acquisition and recognition of social signals. OpenSense enables precisely synchronized and coordinated acquisition and processing of human behavioral signals. Powered by the Microsoft's Platform for Situated Intelligence, OpenSense supports a range of sensor devices and machine learning tools and encourages developers to add new components to the system through straightforward mechanisms for component integration. This platform also offers an intuitive graphical user interface to build application pipelines from existing components. OpenSense is freely available for academic research.

KW - behavior

KW - multimodal

KW - open source

KW - perception

KW - platform

UR - http://www.scopus.com/inward/record.url?scp=85096717332&partnerID=8YFLogxK

U2 - 10.1145/3382507.3418832

DO - 10.1145/3382507.3418832

M3 - Conference Paper

SP - 660

EP - 664

BT - Proceedings of the 2020 International Conference on Multimodal Interaction

A2 - Berthouze, Nadia

A2 - Chetouani, Mohamed

A2 - Nakano, Mikio

PB - Association for Computing Machinery (ACM)

CY - New York NY USA

Y2 - 25 October 2020 through 29 October 2020

ER -