Meta-transfer learning for emotion recognition

Dung Nguyen, Duc Thanh Nguyen, Sridha Sridharan, Simon Denman, Thanh Thi Nguyen, David Dean, Clinton Fookes

Research output: Contribution to journalArticleResearchpeer-review

14 Citations (Scopus)

Abstract

Deep learning has been widely adopted in automatic emotion recognition and has lead to significant progress in the field. However, due to insufficient training data, pre-trained models are limited in their generalisation ability, leading to poor performance on novel test sets. To mitigate this challenge, transfer learning performed by fine-tuning pr-etrained models on novel domains has been applied. However, the fine-tuned knowledge may overwrite and/or discard important knowledge learnt in pre-trained models. In this paper, we address this issue by proposing a PathNet-based meta-transfer learning method that is able to (i) transfer emotional knowledge learnt from one visual/audio emotion domain to another domain and (ii) transfer emotional knowledge learnt from multiple audio emotion domains to one another to improve overall emotion recognition accuracy. To show the robustness of our proposed method, extensive experiments on facial expression-based emotion recognition and speech emotion recognition are carried out on three bench-marking data sets: SAVEE, EMODB, and eNTERFACE. Experimental results show that our proposed method achieves superior performance compared with existing transfer learning methods.

Original languageEnglish
Pages (from-to)10535-10549
Number of pages15
JournalNeural Computing and Applications
Volume35
Issue number14
DOIs
Publication statusPublished - May 2023
Externally publishedYes

Keywords

  • Emotion recognition
  • Facial expression-based emotion recognition
  • Speech emotion recognition
  • Transfer learning

Cite this