Effective recognition of facial micro-expressions with video motion magnification

Yandan Wang, John See, Yee Hui Oh, Raphael C.W. Phan, Yogachandran Rahulamathavan, Huo Chong Ling, Su Wei Tan, Xujie Li

Research output: Contribution to journalArticleResearchpeer-review

52 Citations (Scopus)

Abstract

Facial expression recognition has been intensively studied for decades, notably by the psychology community and more recently the pattern recognition community. What is more challenging, and the subject of more recent research, is the problem of recognizing subtle emotions exhibited by so-called micro-expressions. Recognizing a micro-expression is substantially more challenging than conventional expression recognition because these micro-expressions are only temporally exhibited in a fraction of a second and involve minute spatial changes. Until now, work in this field is at a nascent stage, with only a few existing micro-expression databases and methods. In this article, we propose a new micro-expression recognition approach based on the Eulerian motion magnification technique, which could reveal the hidden information and accentuate the subtle changes in micro-expression motion. Validation of our proposal was done on the recently proposed CASME II dataset in comparison with baseline and state-of-the-art methods. We achieve a good recognition accuracy of up to 75.30 % by using leave-one-out cross validation evaluation protocol. Extensive experiments on various factors at play further demonstrate the effectiveness of our proposed approach.

Original languageEnglish
Pages (from-to)21665-21690
Number of pages26
JournalMultimedia Tools and Applications
Volume76
Issue number20
DOIs
Publication statusPublished - Oct 2017
Externally publishedYes

Keywords

  • CASME II
  • EVM
  • Local binary patterns
  • Micro-expressions
  • Motion magnification

Cite this