Exemplar Hidden Markov Models for classification of facial expressions in videos

Karan Sikka, Abhinav Dhall, Marian Bartlett

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearch

Abstract

Facial expressions are dynamic events comprised of meaningful temporal segments. A common approach to facial expression recognition in video is to first convert variable-length expression sequences into a vector representation by computing summary statistics of image-level features or of spatio-temporal features. These representations are then passed to a discriminative classifier such as a support vector machines (SVM). However, these approaches don't fully exploit the temporal dynamics of facial expressions. Hidden Markov Models (HMMs), provide a method for modeling variable-length expression time-series. Although HMMs have been explored in the past for expression classification, they are rarely used since classification performance is often lower than discriminative approaches, which may be attributed to the challenges of estimating generative models. This paper explores an approach for combining the modeling strength of HMMs with the discriminative power of SVMs via a model-based similarity framework. Each example is first instantiated into an Exemplar-HMM model. A probabilistic kernel is then used to compute a kernel matrix, to be used along with an SVM classifier. This paper proposes that dynamical models such as HMMs are advantageous for the facial expression problem space, when employed in a discriminative, exemplar-based classification framework. The approach yields state-of-the-art results on both posed (CK+ and OULU-CASIA) and spontaneous (FEEDTUM and AM-FED) expression datasets highlighting the performance advantages of the approach.

Original languageEnglish
Title of host publication2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW 2015)
EditorsKristen Grauman, Erik Learned-Miller, Antonio Torralba, Andrew Zisserman
Place of PublicationPiscataway NJ USA
PublisherIEEE, Institute of Electrical and Electronics Engineers
Pages18-25
Number of pages8
ISBN (Electronic)9781467367592, 9781467367585, 9781467367608
ISBN (Print)9781467367592
DOIs
Publication statusPublished - 2015
Externally publishedYes
EventIEEE Conference on Computer Vision and Pattern Recognition Workshops 2015 - Boston, United States of America
Duration: 11 Jun 201512 Jun 2015
http://www.pamitc.org/cvpr15/

Publication series

NameIEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops
PublisherIEEE
ISSN (Print)2160-7508
ISSN (Electronic)2160-7516

Conference

ConferenceIEEE Conference on Computer Vision and Pattern Recognition Workshops 2015
Abbreviated titleCVPRW 2015
CountryUnited States of America
CityBoston
Period11/06/1512/06/15
Internet address

Keywords

  • Computational modeling
  • Hidden Markov models
  • Kernel
  • Probabilistic logic
  • Probability distribution
  • Support vector machines
  • Videos

Cite this

Sikka, K., Dhall, A., & Bartlett, M. (2015). Exemplar Hidden Markov Models for classification of facial expressions in videos. In K. Grauman, E. Learned-Miller, A. Torralba, & A. Zisserman (Eds.), 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW 2015) (pp. 18-25). [7301350] (IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops). IEEE, Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/CVPRW.2015.7301350