Diagnosis of depression by behavioural signals: A multimodal approach

Nicholas Cummins, Jyoti Joshi, Abhinav Dhall, Vidhyasaharan Sethu, Roland Goecke, Julien Epps

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

54 Citations (Scopus)

Abstract

Quantifying behavioural changes in depression using affective computing techniques is the first step in developing an objective diagnostic aid, with clinical utility, for clinical depression. As part of the AVEC 2013 Challenge, we present a multimodal approach for the Depression Sub-Challenge using a GMM-UBM system with three different kernels for the audio subsystem and Space Time Interest Points in a Bag-of-Words approach for the vision subsystem. These are then fused at the feature level to form the combined AV system. Key results include the strong performance of acoustic audio features and the bag-of-words visual features in predicting an individual's level of depression using regression. Interestingly, in the context of the small amount of literature on the subject, is that our feature level multimodal fusion technique is able to outperform both the audio and visual challenge baselines. ©

Original languageEnglish
Title of host publicationAVEC 2013 - Proceedings of the 3rd ACM International Workshop on Audio/Visual Emotion Challenge
PublisherAssociation for Computing Machinery (ACM)
Pages11-20
Number of pages10
ISBN (Print)9781450323956
DOIs
Publication statusPublished - 1 Jan 2013
Externally publishedYes
EventACM International Workshop on Audio/Visual Emotion Challenge, AVEC 2013 - Barcelona, Spain
Duration: 21 Oct 201321 Oct 2013
Conference number: 3rd

Publication series

NameAVEC 2013 - Proceedings of the 3rd ACM International Workshop on Audio/Visual Emotion Challenge

Conference

ConferenceACM International Workshop on Audio/Visual Emotion Challenge, AVEC 2013
CountrySpain
CityBarcelona
Period21/10/1321/10/13

Keywords

  • Acoustic speech features
  • Bag-of-words
  • Behavioural signals
  • Depression
  • Multimodal fusion
  • Multimodal technologies
  • Pyramid of histogram of gradients
  • Space-time interest points
  • Support vector regression

Cite this

Cummins, N., Joshi, J., Dhall, A., Sethu, V., Goecke, R., & Epps, J. (2013). Diagnosis of depression by behavioural signals: A multimodal approach. In AVEC 2013 - Proceedings of the 3rd ACM International Workshop on Audio/Visual Emotion Challenge (pp. 11-20). (AVEC 2013 - Proceedings of the 3rd ACM International Workshop on Audio/Visual Emotion Challenge). Association for Computing Machinery (ACM). https://doi.org/10.1145/2512530.2512535