TY - JOUR
T1 - Meta-transfer learning for emotion recognition
AU - Nguyen, Dung
AU - Nguyen, Duc Thanh
AU - Sridharan, Sridha
AU - Denman, Simon
AU - Nguyen, Thanh Thi
AU - Dean, David
AU - Fookes, Clinton
N1 - Publisher Copyright:
© 2023, The Author(s).
PY - 2023/5
Y1 - 2023/5
N2 - Deep learning has been widely adopted in automatic emotion recognition and has lead to significant progress in the field. However, due to insufficient training data, pre-trained models are limited in their generalisation ability, leading to poor performance on novel test sets. To mitigate this challenge, transfer learning performed by fine-tuning pr-etrained models on novel domains has been applied. However, the fine-tuned knowledge may overwrite and/or discard important knowledge learnt in pre-trained models. In this paper, we address this issue by proposing a PathNet-based meta-transfer learning method that is able to (i) transfer emotional knowledge learnt from one visual/audio emotion domain to another domain and (ii) transfer emotional knowledge learnt from multiple audio emotion domains to one another to improve overall emotion recognition accuracy. To show the robustness of our proposed method, extensive experiments on facial expression-based emotion recognition and speech emotion recognition are carried out on three bench-marking data sets: SAVEE, EMODB, and eNTERFACE. Experimental results show that our proposed method achieves superior performance compared with existing transfer learning methods.
AB - Deep learning has been widely adopted in automatic emotion recognition and has lead to significant progress in the field. However, due to insufficient training data, pre-trained models are limited in their generalisation ability, leading to poor performance on novel test sets. To mitigate this challenge, transfer learning performed by fine-tuning pr-etrained models on novel domains has been applied. However, the fine-tuned knowledge may overwrite and/or discard important knowledge learnt in pre-trained models. In this paper, we address this issue by proposing a PathNet-based meta-transfer learning method that is able to (i) transfer emotional knowledge learnt from one visual/audio emotion domain to another domain and (ii) transfer emotional knowledge learnt from multiple audio emotion domains to one another to improve overall emotion recognition accuracy. To show the robustness of our proposed method, extensive experiments on facial expression-based emotion recognition and speech emotion recognition are carried out on three bench-marking data sets: SAVEE, EMODB, and eNTERFACE. Experimental results show that our proposed method achieves superior performance compared with existing transfer learning methods.
KW - Emotion recognition
KW - Facial expression-based emotion recognition
KW - Speech emotion recognition
KW - Transfer learning
UR - http://www.scopus.com/inward/record.url?scp=85146811394&partnerID=8YFLogxK
U2 - 10.1007/s00521-023-08248-y
DO - 10.1007/s00521-023-08248-y
M3 - Article
AN - SCOPUS:85146811394
SN - 0941-0643
VL - 35
SP - 10535
EP - 10549
JO - Neural Computing and Applications
JF - Neural Computing and Applications
IS - 14
ER -