Multimodal decomposition for enhanced subtle emotion recognition

Zun-Ci Lee, Raphael C.-W. Phan, Su-Wei Tan, Kuan-Heng Lee

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearch

1 Citation (Scopus)


True human emotions may be revealed during brief moments of minute facial muscle movements. These so-called subtle emotions are low in intensity, sparse and short duration in nature. Despite such tiny presence, subtle emotions have shown to be useful for real-life applications including lie detection and clinical diagnosis. In this paper, we present a processing technique based on multimodal decomposition and emotion magnification that is viable to decompose and subsequently emphasize highly discriminative subtle emotions information. Through this method, promising results of 25-30% improvement in recognition performance are achieved. By further magnifying the decomposed emotion components, the subtle emotions intensity can be increased and further boost recognition performance by 3-4%.

Original languageEnglish
Title of host publicationProceedings - Ninth Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, APSIPA ASC 2017
EditorsChang-Su Kim, Wai Lam Hoo
Place of PublicationPiscataway NJ USA
PublisherIEEE, Institute of Electrical and Electronics Engineers
Number of pages7
ISBN (Electronic)9781538615423, 9781538615416
ISBN (Print)9781538615430
Publication statusPublished - 2017
Externally publishedYes
EventAnnual Summit and Conference of the Asia-Pacific-Signal-and-Information-Processing-Association (APSIPA) 2017 - Kuala Lumpur, Malaysia
Duration: 12 Dec 201715 Dec 2017
Conference number: 9th (Proceedings)


ConferenceAnnual Summit and Conference of the Asia-Pacific-Signal-and-Information-Processing-Association (APSIPA) 2017
Abbreviated titleAPSIPA ASC 2017
CityKuala Lumpur
Internet address


  • Decomposition
  • Intensity
  • Magnification
  • Micro-expression recognition
  • Subtle emotion

Cite this