Word-level emotion recognition using high-level features

Johanna D. Moore, Leimin Tian, Catherine Lai

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

22 Citations (Scopus)


In this paper, we investigate the use of high-level features for recognizing human emotions at the word-level in natural conversations with virtual agents. Experiments were carried out on the 2012 Audio/Visual Emotion Challenge (AVEC2012) database, where emotions are defined as vectors in the Arousal-Expectancy-Power-Valence emotional space. Our model using 6 novel disfluency features yields significant improvements compared to those using large number of low-level spectral and prosodic features, and the overall performance difference between it and the best model of the AVEC2012 Word-Level Sub-Challenge is not significant. Our visual model using the Active Shape Model visual features also yields significant improvements compared to models using the low-level Local Binary Patterns visual features. We built a bimodal model By combining our disfluency and visual feature sets and applying Correlation-based Feature-subset Selection. Considering overall performance on all emotion dimensions, our bimodal model outperforms the second best model of the challenge, and comes close to the best model. It also gives the best result when predicting Expectancy values.

Original languageEnglish
Title of host publicationComputational Linguistics and Intelligent Text Processing
Subtitle of host publication15th International Conference, CICLing 2014, Kathmandu, Nepal, April 6-12, 2014, Proceedings, Part II
EditorsAlexander Gelbukh
Place of PublicationBerlin Germany
Number of pages15
ISBN (Electronic)9783642549038
ISBN (Print)9783642549021
Publication statusPublished - 2014
Externally publishedYes
EventInternational Conference on Intelligent Text Processing and Computational Linguistics 2014
- Kathmandu, Nepal
Duration: 6 Apr 201412 Apr 2014
Conference number: 15th

Publication series

NameLecture Notes in Computer Science
NumberPart II
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


ConferenceInternational Conference on Intelligent Text Processing and Computational Linguistics 2014
Abbreviated titleCICLing 2014
Internet address

Cite this