Recognizing emotions in dialogues with disfluencies and non-verbal vocalisations

Leimin Tian; Catherine Lai; Johanna D. Moore

Recognizing emotions in dialogues with disfluencies and non-verbal vocalisations

Leimin Tian, Catherine Lai, Johanna D. Moore

Research output: Chapter in Book/Report/Conference proceeding › Conference Paper › Other

Abstract

We investigate the usefulness of DISfluencies and Non-verbal Vocalisations (DIS-NV) for recognizing human emotions in dialogues. The proposed features measure filled pauses, fillers, stutters, laughter, and breath in utterances. The predictiveness of DISNV features is compared with lexical features and state-of-the-art low-level acoustic features. Our experimental results show that using DIS-NV features alone is not as predictive as using lexical or acoustic features. However, adding them to lexical or acoustic feature set yields improvement compared to using lexical or acoustic features alone. This indicates that disfluencies and non-verbal vocalisations provide useful information overlooked by the other two types of features for emotion recognition.

Original language	English
Title of host publication	Proceedings of the 4th Interdisciplinary Workshop on Laughter and Other Non-verbal Vocalisations in Speech 2015
Subtitle of host publication	14–15 April 2015
Editors	Khiet Truong, Dirk Heylen, Jürgen Trouvain, Nick Campbell
Place of Publication	Piscataway NJ USA
Publisher	IEEE, Institute of Electrical and Electronics Engineers
Pages	39-41
Number of pages	3
Publication status	Published - 2015
Externally published	Yes
Event	Interdisciplinary Workshop on Laughter and Other Non-verbal Vocalisations in Speech 2015 - Enschede, Netherlands Duration: 14 Apr 2015 → 15 Apr 2015 Conference number: 4th https://laughterworkshop2015.wordpress.com/

Conference

Conference	Interdisciplinary Workshop on Laughter and Other Non-verbal Vocalisations in Speech 2015
Country/Territory	Netherlands
City	Enschede
Period	14/04/15 → 15/04/15
Internet address	https://laughterworkshop2015.wordpress.com/

Keywords

emotion recognition
dialogue
disfluency
speech processing
HCI

Access to Document

257088422-oaFinal published version, 61.1 KB

Cite this

Tian, L., Lai, C., & Moore, J. D. (2015). Recognizing emotions in dialogues with disfluencies and non-verbal vocalisations. In K. Truong, D. Heylen, J. Trouvain, & N. Campbell (Eds.), Proceedings of the 4th Interdisciplinary Workshop on Laughter and Other Non-verbal Vocalisations in Speech 2015: 14–15 April 2015 (pp. 39-41). IEEE, Institute of Electrical and Electronics Engineers.

Tian, Leimin ; Lai, Catherine ; Moore, Johanna D. / Recognizing emotions in dialogues with disfluencies and non-verbal vocalisations. Proceedings of the 4th Interdisciplinary Workshop on Laughter and Other Non-verbal Vocalisations in Speech 2015: 14–15 April 2015. editor / Khiet Truong ; Dirk Heylen ; Jürgen Trouvain ; Nick Campbell. Piscataway NJ USA : IEEE, Institute of Electrical and Electronics Engineers, 2015. pp. 39-41

@inproceedings{d311b268b50543c497499055b3666c30,

title = "Recognizing emotions in dialogues with disfluencies and non-verbal vocalisations",

abstract = "We investigate the usefulness of DISfluencies and Non-verbal Vocalisations (DIS-NV) for recognizing human emotions in dialogues. The proposed features measure filled pauses, fillers, stutters, laughter, and breath in utterances. The predictiveness of DISNV features is compared with lexical features and state-of-the-art low-level acoustic features. Our experimental results show that using DIS-NV features alone is not as predictive as using lexical or acoustic features. However, adding them to lexical or acoustic feature set yields improvement compared to using lexical or acoustic features alone. This indicates that disfluencies and non-verbal vocalisations provide useful information overlooked by the other two types of features for emotion recognition.",

keywords = "emotion recognition, dialogue, disfluency, speech processing, HCI",

author = "Leimin Tian and Catherine Lai and Moore, {Johanna D.}",

year = "2015",

language = "English",

pages = "39--41",

editor = "Truong, {Khiet } and Heylen, {Dirk } and Trouvain, {J{\"u}rgen } and Nick Campbell",

booktitle = "Proceedings of the 4th Interdisciplinary Workshop on Laughter and Other Non-verbal Vocalisations in Speech 2015",

publisher = "IEEE, Institute of Electrical and Electronics Engineers",

address = "United States of America",

note = "Interdisciplinary Workshop on Laughter and Other Non-verbal Vocalisations in Speech 2015 ; Conference date: 14-04-2015 Through 15-04-2015",

url = "https://laughterworkshop2015.wordpress.com/",

}

Tian, L, Lai, C & Moore, JD 2015, Recognizing emotions in dialogues with disfluencies and non-verbal vocalisations. in K Truong, D Heylen, J Trouvain & N Campbell (eds), Proceedings of the 4th Interdisciplinary Workshop on Laughter and Other Non-verbal Vocalisations in Speech 2015: 14–15 April 2015. IEEE, Institute of Electrical and Electronics Engineers, Piscataway NJ USA, pp. 39-41, Interdisciplinary Workshop on Laughter and Other Non-verbal Vocalisations in Speech 2015, Enschede, Netherlands, 14/04/15.

Recognizing emotions in dialogues with disfluencies and non-verbal vocalisations. / Tian, Leimin; Lai, Catherine; Moore, Johanna D.
Proceedings of the 4th Interdisciplinary Workshop on Laughter and Other Non-verbal Vocalisations in Speech 2015: 14–15 April 2015. ed. / Khiet Truong; Dirk Heylen; Jürgen Trouvain; Nick Campbell. Piscataway NJ USA: IEEE, Institute of Electrical and Electronics Engineers, 2015. p. 39-41.

Research output: Chapter in Book/Report/Conference proceeding › Conference Paper › Other

TY - GEN

T1 - Recognizing emotions in dialogues with disfluencies and non-verbal vocalisations

AU - Tian, Leimin

AU - Lai, Catherine

AU - Moore, Johanna D.

N1 - Conference code: 4th

PY - 2015

Y1 - 2015

N2 - We investigate the usefulness of DISfluencies and Non-verbal Vocalisations (DIS-NV) for recognizing human emotions in dialogues. The proposed features measure filled pauses, fillers, stutters, laughter, and breath in utterances. The predictiveness of DISNV features is compared with lexical features and state-of-the-art low-level acoustic features. Our experimental results show that using DIS-NV features alone is not as predictive as using lexical or acoustic features. However, adding them to lexical or acoustic feature set yields improvement compared to using lexical or acoustic features alone. This indicates that disfluencies and non-verbal vocalisations provide useful information overlooked by the other two types of features for emotion recognition.

AB - We investigate the usefulness of DISfluencies and Non-verbal Vocalisations (DIS-NV) for recognizing human emotions in dialogues. The proposed features measure filled pauses, fillers, stutters, laughter, and breath in utterances. The predictiveness of DISNV features is compared with lexical features and state-of-the-art low-level acoustic features. Our experimental results show that using DIS-NV features alone is not as predictive as using lexical or acoustic features. However, adding them to lexical or acoustic feature set yields improvement compared to using lexical or acoustic features alone. This indicates that disfluencies and non-verbal vocalisations provide useful information overlooked by the other two types of features for emotion recognition.

KW - emotion recognition

KW - dialogue

KW - disfluency

KW - speech processing

KW - HCI

M3 - Conference Paper

SP - 39

EP - 41

BT - Proceedings of the 4th Interdisciplinary Workshop on Laughter and Other Non-verbal Vocalisations in Speech 2015

A2 - Truong, Khiet

A2 - Heylen, Dirk

A2 - Trouvain, Jürgen

A2 - Campbell, Nick

PB - IEEE, Institute of Electrical and Electronics Engineers

CY - Piscataway NJ USA

T2 - Interdisciplinary Workshop on Laughter and Other Non-verbal Vocalisations in Speech 2015

Y2 - 14 April 2015 through 15 April 2015

ER -

Tian L, Lai C, Moore JD. Recognizing emotions in dialogues with disfluencies and non-verbal vocalisations. In Truong K, Heylen D, Trouvain J, Campbell N, editors, Proceedings of the 4th Interdisciplinary Workshop on Laughter and Other Non-verbal Vocalisations in Speech 2015: 14–15 April 2015. Piscataway NJ USA: IEEE, Institute of Electrical and Electronics Engineers. 2015. p. 39-41