Dual-stream shallow networks for facial micro-expression recognition

Huai-Qian Khor, John See, Sze-Teng Liong, Raphael C.W. Phan, Weiyao Lin

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

106 Citations (Scopus)


Micro-expressions are spontaneous, brief and subtle facial muscle movements that exposes underlying emotions. Motivated by recent exploits into deep learning for micro-expression analysis, we propose a lightweight dual-stream shallow network in the form of a pair of truncated CNNs with heterogeneous input features. The merging of the convolutional features allows for discriminative learning of micro-expression classes stemming from both streams. Using activation heatmaps, we further demonstrate that salient facial areas are well emphasized, and correspond closely to relevant action units belonging to emotion classes. We empirically validate the proposed network on three benchmark databases, obtaining state-of-the-art performance on the CASME II and SAMM while remaining competitive on the SMIC. Further observations point towards the sufficiency of utilizing shallower deep networks for micro-expression recognition.

Original languageEnglish
Title of host publication2019 IEEE International Conference on Image Processing - Proceedings
EditorsWen-Hsiao Peng
Place of PublicationPiscataway NJ USA
PublisherIEEE, Institute of Electrical and Electronics Engineers
Number of pages5
ISBN (Electronic)9781538662496
ISBN (Print)9781538662502
Publication statusPublished - 2019
EventIEEE International Conference on Image Processing 2019 - Taipei, Taiwan
Duration: 22 Sept 201925 Sept 2019
Conference number: 26th
https://ieeexplore.ieee.org/xpl/conhome/8791230/proceeding (Proceedings)

Publication series

NameProceedings - International Conference on Image Processing, ICIP
PublisherThe Institute of Electrical and Electronics Engineers, Inc.
ISSN (Print)1522-4880


ConferenceIEEE International Conference on Image Processing 2019
Abbreviated titleICIP 2019
Internet address


  • apex frame
  • dual-stream networks
  • Micro-expression
  • recognition
  • shallow CNN

Cite this