Diagnosis of depression by behavioural signals: A multimodal approach

Nicholas Cummins; Jyoti Joshi; Abhinav Dhall; Vidhyasaharan Sethu; Roland Goecke; Julien Epps

doi:10.1145/2512530.2512535

Diagnosis of depression by behavioural signals: A multimodal approach

Nicholas Cummins, Jyoti Joshi, Abhinav Dhall, Vidhyasaharan Sethu, Roland Goecke, Julien Epps

Research output: Chapter in Book/Report/Conference proceeding › Conference Paper › Research › peer-review

112 Citations (Scopus)

Abstract

Quantifying behavioural changes in depression using affective computing techniques is the first step in developing an objective diagnostic aid, with clinical utility, for clinical depression. As part of the AVEC 2013 Challenge, we present a multimodal approach for the Depression Sub-Challenge using a GMM-UBM system with three different kernels for the audio subsystem and Space Time Interest Points in a Bag-of-Words approach for the vision subsystem. These are then fused at the feature level to form the combined AV system. Key results include the strong performance of acoustic audio features and the bag-of-words visual features in predicting an individual's level of depression using regression. Interestingly, in the context of the small amount of literature on the subject, is that our feature level multimodal fusion technique is able to outperform both the audio and visual challenge baselines. ©

Original language	English
Title of host publication	AVEC 2013 - Proceedings of the 3rd ACM International Workshop on Audio/Visual Emotion Challenge
Publisher	Association for Computing Machinery (ACM)
Pages	11-20
Number of pages	10
ISBN (Print)	9781450323956
DOIs	https://doi.org/10.1145/2512530.2512535
Publication status	Published - 1 Jan 2013
Externally published	Yes
Event	ACM International Workshop on Audio/Visual Emotion Challenge, AVEC 2013 - Barcelona, Spain Duration: 21 Oct 2013 → 21 Oct 2013 Conference number: 3rd

Publication series

Name	AVEC 2013 - Proceedings of the 3rd ACM International Workshop on Audio/Visual Emotion Challenge

Conference

Conference	ACM International Workshop on Audio/Visual Emotion Challenge, AVEC 2013
Country/Territory	Spain
City	Barcelona
Period	21/10/13 → 21/10/13

Keywords

Acoustic speech features
Bag-of-words
Behavioural signals
Depression
Multimodal fusion
Multimodal technologies
Pyramid of histogram of gradients
Space-time interest points
Support vector regression

Access to Document

10.1145/2512530.2512535

Cite this

Cummins, N., Joshi, J., Dhall, A., Sethu, V., Goecke, R., & Epps, J. (2013). Diagnosis of depression by behavioural signals: A multimodal approach. In AVEC 2013 - Proceedings of the 3rd ACM International Workshop on Audio/Visual Emotion Challenge (pp. 11-20). (AVEC 2013 - Proceedings of the 3rd ACM International Workshop on Audio/Visual Emotion Challenge). Association for Computing Machinery (ACM). https://doi.org/10.1145/2512530.2512535

Cummins, Nicholas ; Joshi, Jyoti ; Dhall, Abhinav et al. / Diagnosis of depression by behavioural signals : A multimodal approach. AVEC 2013 - Proceedings of the 3rd ACM International Workshop on Audio/Visual Emotion Challenge. Association for Computing Machinery (ACM), 2013. pp. 11-20 (AVEC 2013 - Proceedings of the 3rd ACM International Workshop on Audio/Visual Emotion Challenge).

@inproceedings{dc627d9d642549769eb0f09a4bf8ed70,

title = "Diagnosis of depression by behavioural signals: A multimodal approach",

abstract = "Quantifying behavioural changes in depression using affective computing techniques is the first step in developing an objective diagnostic aid, with clinical utility, for clinical depression. As part of the AVEC 2013 Challenge, we present a multimodal approach for the Depression Sub-Challenge using a GMM-UBM system with three different kernels for the audio subsystem and Space Time Interest Points in a Bag-of-Words approach for the vision subsystem. These are then fused at the feature level to form the combined AV system. Key results include the strong performance of acoustic audio features and the bag-of-words visual features in predicting an individual's level of depression using regression. Interestingly, in the context of the small amount of literature on the subject, is that our feature level multimodal fusion technique is able to outperform both the audio and visual challenge baselines. {\textcopyright}",

keywords = "Acoustic speech features, Bag-of-words, Behavioural signals, Depression, Multimodal fusion, Multimodal technologies, Pyramid of histogram of gradients, Space-time interest points, Support vector regression",

author = "Nicholas Cummins and Jyoti Joshi and Abhinav Dhall and Vidhyasaharan Sethu and Roland Goecke and Julien Epps",

year = "2013",

month = jan,

day = "1",

doi = "10.1145/2512530.2512535",

language = "English",

isbn = "9781450323956",

series = "AVEC 2013 - Proceedings of the 3rd ACM International Workshop on Audio/Visual Emotion Challenge",

publisher = "Association for Computing Machinery (ACM)",

pages = "11--20",

booktitle = "AVEC 2013 - Proceedings of the 3rd ACM International Workshop on Audio/Visual Emotion Challenge",

address = "United States of America",

note = "ACM International Workshop on Audio/Visual Emotion Challenge, AVEC 2013 ; Conference date: 21-10-2013 Through 21-10-2013",

}

Cummins, N, Joshi, J, Dhall, A, Sethu, V, Goecke, R & Epps, J 2013, Diagnosis of depression by behavioural signals: A multimodal approach. in AVEC 2013 - Proceedings of the 3rd ACM International Workshop on Audio/Visual Emotion Challenge. AVEC 2013 - Proceedings of the 3rd ACM International Workshop on Audio/Visual Emotion Challenge, Association for Computing Machinery (ACM), pp. 11-20, ACM International Workshop on Audio/Visual Emotion Challenge, AVEC 2013, Barcelona, Spain, 21/10/13. https://doi.org/10.1145/2512530.2512535

Diagnosis of depression by behavioural signals: A multimodal approach. / Cummins, Nicholas; Joshi, Jyoti; Dhall, Abhinav et al.
AVEC 2013 - Proceedings of the 3rd ACM International Workshop on Audio/Visual Emotion Challenge. Association for Computing Machinery (ACM), 2013. p. 11-20 (AVEC 2013 - Proceedings of the 3rd ACM International Workshop on Audio/Visual Emotion Challenge).

Research output: Chapter in Book/Report/Conference proceeding › Conference Paper › Research › peer-review

TY - GEN

T1 - Diagnosis of depression by behavioural signals

T2 - ACM International Workshop on Audio/Visual Emotion Challenge, AVEC 2013

AU - Cummins, Nicholas

AU - Joshi, Jyoti

AU - Dhall, Abhinav

AU - Sethu, Vidhyasaharan

AU - Goecke, Roland

AU - Epps, Julien

N1 - Conference code: 3rd

PY - 2013/1/1

Y1 - 2013/1/1

N2 - Quantifying behavioural changes in depression using affective computing techniques is the first step in developing an objective diagnostic aid, with clinical utility, for clinical depression. As part of the AVEC 2013 Challenge, we present a multimodal approach for the Depression Sub-Challenge using a GMM-UBM system with three different kernels for the audio subsystem and Space Time Interest Points in a Bag-of-Words approach for the vision subsystem. These are then fused at the feature level to form the combined AV system. Key results include the strong performance of acoustic audio features and the bag-of-words visual features in predicting an individual's level of depression using regression. Interestingly, in the context of the small amount of literature on the subject, is that our feature level multimodal fusion technique is able to outperform both the audio and visual challenge baselines. ©

AB - Quantifying behavioural changes in depression using affective computing techniques is the first step in developing an objective diagnostic aid, with clinical utility, for clinical depression. As part of the AVEC 2013 Challenge, we present a multimodal approach for the Depression Sub-Challenge using a GMM-UBM system with three different kernels for the audio subsystem and Space Time Interest Points in a Bag-of-Words approach for the vision subsystem. These are then fused at the feature level to form the combined AV system. Key results include the strong performance of acoustic audio features and the bag-of-words visual features in predicting an individual's level of depression using regression. Interestingly, in the context of the small amount of literature on the subject, is that our feature level multimodal fusion technique is able to outperform both the audio and visual challenge baselines. ©

KW - Acoustic speech features

KW - Bag-of-words

KW - Behavioural signals

KW - Depression

KW - Multimodal fusion

KW - Multimodal technologies

KW - Pyramid of histogram of gradients

KW - Space-time interest points

KW - Support vector regression

UR - http://www.scopus.com/inward/record.url?scp=84887498641&partnerID=8YFLogxK

U2 - 10.1145/2512530.2512535

DO - 10.1145/2512530.2512535

M3 - Conference Paper

AN - SCOPUS:84887498641

SN - 9781450323956

T3 - AVEC 2013 - Proceedings of the 3rd ACM International Workshop on Audio/Visual Emotion Challenge

SP - 11

EP - 20

BT - AVEC 2013 - Proceedings of the 3rd ACM International Workshop on Audio/Visual Emotion Challenge

PB - Association for Computing Machinery (ACM)

Y2 - 21 October 2013 through 21 October 2013

ER -

Cummins N, Joshi J, Dhall A, Sethu V, Goecke R, Epps J. Diagnosis of depression by behavioural signals: A multimodal approach. In AVEC 2013 - Proceedings of the 3rd ACM International Workshop on Audio/Visual Emotion Challenge. Association for Computing Machinery (ACM). 2013. p. 11-20. (AVEC 2013 - Proceedings of the 3rd ACM International Workshop on Audio/Visual Emotion Challenge). doi: 10.1145/2512530.2512535

Diagnosis of depression by behavioural signals: A multimodal approach

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Cite this