Automatic group affect analysis in images via visual attribute and feature networks

Shreya Ghosh, Abhinav Dhall, Nicu Sebe

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

27 Citations (Scopus)


This paper proposes a pipeline for automatic group-level affect analysis. A deep neural network-based approach, which leverages on the facial-expression information, scene information and a high-level facial visual attribute information is proposed. A capsule network-based architecture is used to predict the facial expression. Transfer learning is used on Inception-V3 to extract global image-based features which contain scene information. Another network is trained for inferring the facial attributes of the group members. Further, these attributes are pooled at a group-level to train a network for inferring the group-level affect. The facial attribute prediction network, although is simple yet, is effective and generates result comparable to the state-of-the-art methods. Later, model integration is performed from the three channels. The experiments show the effectiveness of the proposed techniques on three 'in the wild' databases: Group Affect Database, HAPPEI and UCLA-Protest database.

Original languageEnglish
Title of host publication2018 IEEE International Conference on Image Processing - Proceedings
EditorsNikolaos Boulgouris, Lisimachos P. Kondi
Place of PublicationPiscataway NJ USA
PublisherIEEE, Institute of Electrical and Electronics Engineers
Number of pages5
ISBN (Electronic)9781479970612
ISBN (Print)9781479970629
Publication statusPublished - 2018
Externally publishedYes
EventIEEE International Conference on Image Processing 2018 - Athens, Greece
Duration: 7 Oct 201810 Oct 2018
Conference number: 25th (Proceedings)


ConferenceIEEE International Conference on Image Processing 2018
Abbreviated titleICIP 2018
Internet address


  • Group level affect recognition

Cite this