Automatic group affect analysis in images via visual attribute and feature networks

Shreya Ghosh, Abhinav Dhall, Nicu Sebe

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

8 Citations (Scopus)

Abstract

This paper proposes a pipeline for automatic group-level affect analysis. A deep neural network-based approach, which leverages on the facial-expression information, scene information and a high-level facial visual attribute information is proposed. A capsule network-based architecture is used to predict the facial expression. Transfer learning is used on Inception-V3 to extract global image-based features which contain scene information. Another network is trained for inferring the facial attributes of the group members. Further, these attributes are pooled at a group-level to train a network for inferring the group-level affect. The facial attribute prediction network, although is simple yet, is effective and generates result comparable to the state-of-the-art methods. Later, model integration is performed from the three channels. The experiments show the effectiveness of the proposed techniques on three 'in the wild' databases: Group Affect Database, HAPPEI and UCLA-Protest database.

Original languageEnglish
Title of host publication2018 IEEE International Conference on Image Processing - Proceedings
EditorsNikolaos Boulgouris, Lisimachos P. Kondi
Place of PublicationPiscataway NJ USA
PublisherIEEE, Institute of Electrical and Electronics Engineers
Pages1967-1971
Number of pages5
ISBN (Electronic)9781479970612
ISBN (Print)9781479970629
DOIs
Publication statusPublished - 2018
Externally publishedYes
EventIEEE International Conference on Image Processing 2018 - Athens, Greece
Duration: 7 Oct 201810 Oct 2018
Conference number: 25th
https://2018.ieeeicip.org/

Conference

ConferenceIEEE International Conference on Image Processing 2018
Abbreviated titleICIP 2018
CountryGreece
CityAthens
Period7/10/1810/10/18
Internet address

Keywords

  • Group level affect recognition

Cite this

Ghosh, S., Dhall, A., & Sebe, N. (2018). Automatic group affect analysis in images via visual attribute and feature networks. In N. Boulgouris, & L. P. Kondi (Eds.), 2018 IEEE International Conference on Image Processing - Proceedings (pp. 1967-1971). IEEE, Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/ICIP.2018.8451242