Automatic micro-expression recognition from long video using a single spotted apex

Sze-Teng Liong, John See, KokSheik Wong, Raphael Chung Wei Phan

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

33 Citations (Scopus)


Recently, micro-expression recognition has seen an increase of interest from psychological and computer vision communities. As micro-expressions are generated involuntarily on a person’s face, and are usually a manifestation of repressed feelings of the person. Most existing works pay attention to either the detection or spotting of micro-expression frames or the categorization of type of micro-expression present in a short video shot. In this paper, we introduced a novel automatic approach to micro-expression recognition from long video that combines both spotting and recognition mechanisms. To achieve this, the apex frame, which provides the instant when the highest intensity of facial movement occurs, is first spotted from the entire video sequence. An automatic eye masking technique is also presented to improve the robustness of apex frame spotting. With the single apex, we describe the spotted micro-expression instant using a state-of-the-art feature extractor before proceeding to classification. This is the first known work that recognizes micro-expressions from a long video sequence without the knowledge of onset and offset frames, which are typically used to determine a cropped sub-sequence containing the micro-expression. We evaluated the spotting and recognition tasks on four spontaneous micro-expression databases comprising only of raw long videos – CASME II-RAW, SMIC-E-HS, SMIC-E-VIS and SMIC-E-NIR. We obtained compelling results that show the effectiveness of the proposed approach, which outperform most methods that rely on human annotated sub-sequences.

Original languageEnglish
Title of host publicationComputer Vision - ACCV 2016 Workshops
Subtitle of host publicationACCV 2016 International Workshops Taipei, Taiwan, November 20–24, 2016 Revised Selected Papers, Part II
EditorsChu-Song Chen, Jiwen Lu, Kai-Kuang Ma
Place of PublicationCham Switzerland
Number of pages16
ISBN (Electronic)9783319544274
ISBN (Print)9783319544267
Publication statusPublished - 2017
Externally publishedYes
EventWorkshop on Facial Informatics 2016 - Taipei, Taiwan
Duration: 20 Nov 201624 Nov 2016 (Proceedings) (Website)

Publication series

NameLecture Notes in Computer Science
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


ConferenceWorkshop on Facial Informatics 2016
Abbreviated titleWFI 2016
City Taipei
Internet address

Cite this