Heart rate estimation from facial videos for depression analysis

Aamir Mustafa, Shalini Bhatia, Munawar Hayat, Roland Goecke

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearch

6 Citations (Scopus)


Automated facial video analysis is useful in numerous health care applications. For example, spatio-temporal analysis of such videos has been previously done for assisting clinicians in the diagnosis of depression. Physiological measures, such as an individual's heart rate, provide very important cues to understand a person's mental health. Unobtrusively estimated heart rate has not been previously used to analyse individuals' mental health. In this paper, we automatically estimate heart rate activity from facial videos. We then study the association of the estimated heart rate activity with the person's mental health, as diagnosed by clinicians. Specifically, from the heart rate activity in response to watching different movies, we classify individuals as either depressed or healthy. The efficacy of the proposed scheme is demonstrated by experimental evaluations on a clinically validated dataset. Our results suggest unobtrusively estimated heart rate to be very effective for depression analysis.

Original languageEnglish
Title of host publicationThe 7th International Conference on Affective Computing and Intelligent Interaction (ACII 2017)
EditorsHayley Hung, Emily Mower Provost, Mohammad Soleymani
Place of PublicationPiscataway NJ USA
PublisherIEEE, Institute of Electrical and Electronics Engineers
Number of pages6
ISBN (Electronic)9781538605639
ISBN (Print)9781538605646
Publication statusPublished - 2017
Externally publishedYes
EventInternational Conference on Affective Computing and Intelligent Interaction 2017 - San Antonio, United States of America
Duration: 23 Oct 201726 Oct 2017
Conference number: 7th


ConferenceInternational Conference on Affective Computing and Intelligent Interaction 2017
Abbreviated titleACII 2017
Country/TerritoryUnited States of America
CitySan Antonio
Internet address

Cite this