ThermalWear: exploring wearable on-chest thermal displays to augment voice messages with affect

Abdallah El Ali; Xingyu Yang; Swamy Ananthanarayan; Thomas Röggla; Jack Jansen; Jess Hartcher-O'Brien; Kaspar Jansen; Pablo Cesar

doi:10.1145/3313831.3376682

ThermalWear: exploring wearable on-chest thermal displays to augment voice messages with affect

Abdallah El Ali, Xingyu Yang, Swamy Ananthanarayan, Thomas Röggla, Jack Jansen, Jess Hartcher-O'Brien, Kaspar Jansen, Pablo Cesar

Research output: Chapter in Book/Report/Conference proceeding › Conference Paper › Research › peer-review

19 Citations (Scopus)

Abstract

Voice is a rich modality for conveying emotions, however emotional prosody production can be situationally or medically impaired. Since thermal displays have been shown to evoke emotions, we explore how thermal stimulation can augment perception of neutrally-spoken voice messages with affect. We designed ThermalWear, a wearable on-chest thermal display, then tested in a controlled study (N=12) the effects of fabric, thermal intensity, and direction of change. Thereafter, we synthesized 12 neutrally-spoken voice messages, validated (N=7) them, then tested (N=12) if thermal stimuli can augment their perception with affect. We found warm and cool stimuli (a) can be perceived on the chest, and quickly without fabric (4.7-5s) (b) do not incur discomfort (c) generally increase arousal of voice messages and (d) increase / decrease message valence, respectively. We discuss how thermal displays can augment voice perception, which can enhance voice assistants and support individuals with emotional prosody impairments.

Original language	English
Title of host publication	Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems
Editors	Joanna McGrenere, Andy Cockburn
Place of Publication	New York NY USA
Publisher	Association for Computing Machinery (ACM)
Number of pages	14
ISBN (Electronic)	9781450367080
DOIs	https://doi.org/10.1145/3313831.3376682
Publication status	Published - 2020
Externally published	Yes
Event	International Conference on Human Factors in Computing Systems 2020 - Honolulu , United States of America Duration: 25 Apr 2020 → 30 Apr 2020 Conference number: 38th https://chi2020.acm.org (Website) https://dl.acm.org/doi/proceedings/10.1145/3313831 (Proceedings)

Conference

Conference	International Conference on Human Factors in Computing Systems 2020
Abbreviated title	CHI 2020
Country/Territory	United States of America
City	Honolulu
Period	25/04/20 → 30/04/20
Internet address	https://chi2020.acm.org (Website) https://dl.acm.org/doi/proceedings/10.1145/3313831 (Proceedings)

Keywords

voice
prosody
affect
display
wearable
thermal
chest
emotion

Access to Document

10.1145/3313831.3376682

Cite this

El Ali, A., Yang, X., Ananthanarayan, S., Röggla, T., Jansen, J., Hartcher-O'Brien, J., Jansen, K., & Cesar, P. (2020). ThermalWear: exploring wearable on-chest thermal displays to augment voice messages with affect. In J. McGrenere, & A. Cockburn (Eds.), Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems [3376682] Association for Computing Machinery (ACM). https://doi.org/10.1145/3313831.3376682

@inproceedings{544a414ac7524eadab1fd81d6e1924d0,

title = "ThermalWear: exploring wearable on-chest thermal displays to augment voice messages with affect",

abstract = "Voice is a rich modality for conveying emotions, however emotional prosody production can be situationally or medically impaired. Since thermal displays have been shown to evoke emotions, we explore how thermal stimulation can augment perception of neutrally-spoken voice messages with affect. We designed ThermalWear, a wearable on-chest thermal display, then tested in a controlled study (N=12) the effects of fabric, thermal intensity, and direction of change. Thereafter, we synthesized 12 neutrally-spoken voice messages, validated (N=7) them, then tested (N=12) if thermal stimuli can augment their perception with affect. We found warm and cool stimuli (a) can be perceived on the chest, and quickly without fabric (4.7-5s) (b) do not incur discomfort (c) generally increase arousal of voice messages and (d) increase / decrease message valence, respectively. We discuss how thermal displays can augment voice perception, which can enhance voice assistants and support individuals with emotional prosody impairments.",

keywords = "voice, prosody, affect, display, wearable, thermal, chest, emotion",

author = "{El Ali}, Abdallah and Xingyu Yang and Swamy Ananthanarayan and Thomas R{\"o}ggla and Jack Jansen and Jess Hartcher-O'Brien and Kaspar Jansen and Pablo Cesar",

year = "2020",

doi = "10.1145/3313831.3376682",

language = "English",

editor = "McGrenere, {Joanna } and Cockburn, {Andy }",

booktitle = "Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems",

publisher = "Association for Computing Machinery (ACM)",

address = "United States of America",

note = "International Conference on Human Factors in Computing Systems 2020, CHI 2020 ; Conference date: 25-04-2020 Through 30-04-2020",

url = "https://chi2020.acm.org, https://dl.acm.org/doi/proceedings/10.1145/3313831",

}

El Ali, A, Yang, X, Ananthanarayan, S, Röggla, T, Jansen, J, Hartcher-O'Brien, J, Jansen, K & Cesar, P 2020, ThermalWear: exploring wearable on-chest thermal displays to augment voice messages with affect. in J McGrenere & A Cockburn (eds), Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems., 3376682, Association for Computing Machinery (ACM), New York NY USA, International Conference on Human Factors in Computing Systems 2020, Honolulu , Hawaii, United States of America, 25/04/20. https://doi.org/10.1145/3313831.3376682

ThermalWear : exploring wearable on-chest thermal displays to augment voice messages with affect. / El Ali, Abdallah; Yang, Xingyu; Ananthanarayan, Swamy et al.

Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. ed. / Joanna McGrenere; Andy Cockburn. New York NY USA : Association for Computing Machinery (ACM), 2020. 3376682.

Research output: Chapter in Book/Report/Conference proceeding › Conference Paper › Research › peer-review

TY - GEN

T1 - ThermalWear

T2 - International Conference on Human Factors in Computing Systems 2020

AU - El Ali, Abdallah

AU - Yang, Xingyu

AU - Ananthanarayan, Swamy

AU - Röggla, Thomas

AU - Jansen, Jack

AU - Hartcher-O'Brien, Jess

AU - Jansen, Kaspar

AU - Cesar, Pablo

N1 - Conference code: 38th

PY - 2020

Y1 - 2020

N2 - Voice is a rich modality for conveying emotions, however emotional prosody production can be situationally or medically impaired. Since thermal displays have been shown to evoke emotions, we explore how thermal stimulation can augment perception of neutrally-spoken voice messages with affect. We designed ThermalWear, a wearable on-chest thermal display, then tested in a controlled study (N=12) the effects of fabric, thermal intensity, and direction of change. Thereafter, we synthesized 12 neutrally-spoken voice messages, validated (N=7) them, then tested (N=12) if thermal stimuli can augment their perception with affect. We found warm and cool stimuli (a) can be perceived on the chest, and quickly without fabric (4.7-5s) (b) do not incur discomfort (c) generally increase arousal of voice messages and (d) increase / decrease message valence, respectively. We discuss how thermal displays can augment voice perception, which can enhance voice assistants and support individuals with emotional prosody impairments.

AB - Voice is a rich modality for conveying emotions, however emotional prosody production can be situationally or medically impaired. Since thermal displays have been shown to evoke emotions, we explore how thermal stimulation can augment perception of neutrally-spoken voice messages with affect. We designed ThermalWear, a wearable on-chest thermal display, then tested in a controlled study (N=12) the effects of fabric, thermal intensity, and direction of change. Thereafter, we synthesized 12 neutrally-spoken voice messages, validated (N=7) them, then tested (N=12) if thermal stimuli can augment their perception with affect. We found warm and cool stimuli (a) can be perceived on the chest, and quickly without fabric (4.7-5s) (b) do not incur discomfort (c) generally increase arousal of voice messages and (d) increase / decrease message valence, respectively. We discuss how thermal displays can augment voice perception, which can enhance voice assistants and support individuals with emotional prosody impairments.

KW - voice

KW - prosody

KW - affect

KW - display

KW - wearable

KW - thermal

KW - chest

KW - emotion

UR - http://www.scopus.com/inward/record.url?scp=85091302749&partnerID=8YFLogxK

U2 - 10.1145/3313831.3376682

DO - 10.1145/3313831.3376682

M3 - Conference Paper

BT - Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems

A2 - McGrenere, Joanna

A2 - Cockburn, Andy

PB - Association for Computing Machinery (ACM)

CY - New York NY USA

Y2 - 25 April 2020 through 30 April 2020

ER -

El Ali A, Yang X, Ananthanarayan S, Röggla T, Jansen J, Hartcher-O'Brien J et al. ThermalWear: exploring wearable on-chest thermal displays to augment voice messages with affect. In McGrenere J, Cockburn A, editors, Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. New York NY USA: Association for Computing Machinery (ACM). 2020. 3376682 https://doi.org/10.1145/3313831.3376682

ThermalWear: exploring wearable on-chest thermal displays to augment voice messages with affect

Abstract

Conference

Keywords

Access to Document

Other files and links

Cite this