Jointly predicting affective and mental health scores using deep neural networks of visual cues on the web

Hung Nguyen, Van Nguyen, Thin Nguyen, Mark E. Larsen, Bridianne O’Dea, Duc Thanh Nguyen, Trung Le, Dinh Phung, Svetha Venkatesh, Helen Christensen

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

Abstract

Despite the range of studies examining the relationship between mental health and social media data, not all prior studies have validated the social media markers against “ground truth”, or validated psychiatric information, in general community samples. Instead, researchers have approximated psychiatric diagnosis using user statements such as “I have been diagnosed as X”. Without “ground truth”, the value of predictive algorithms is highly questionable and potentially harmful. In addition, for social media data, whilst linguistic features have been widely identified as strong markers of mental health disorders, little is known about non-textual features on their links with the disorders. The current work is a longitudinal study during which participants’ mental health data, consisting of depression and anxiety scores, were collected fortnightly with a validated, diagnostic, clinical measure. Also, datasets with labels relevant to mental health scores, such as emotional scores, are also employed to improve the performance in prediction of mental health scores. This work introduces a deep neural network-based method integrating sub-networks on predicting affective scores and mental health outcomes from images. Experimental results have shown that in the both predictions of emotion and mental health scores, (1) deep features majorly outperform handcrafted ones and (2) the proposed network achieves better performance compared with separate networks.

Original languageEnglish
Title of host publicationWeb Information Systems Engineering – WISE 2018
Subtitle of host publication19th International Conference Dubai, United Arab Emirates, November 12–15, 2018 Proceedings, Part II
EditorsHakim Hacid, Wojciech Cellary, Hua Wang, Hye-Young Paik, Rui Zhou
Place of PublicationCham Switzerland
PublisherSpringer
Pages100-110
Number of pages11
ISBN (Electronic)9783030029258
ISBN (Print)9783030029241
DOIs
Publication statusPublished - 2018
EventInternational Conference on Web Information Systems Engineering 2018 - Dubai, United Arab Emirates
Duration: 12 Nov 201815 Nov 2018
Conference number: 19th
http://wise2018.connect.rs/index.html

Publication series

NameLecture Notes in Computer Science
PublisherSpringer
Volume11234
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

ConferenceInternational Conference on Web Information Systems Engineering 2018
Abbreviated titleWISE 2018
CountryUnited Arab Emirates
CityDubai
Period12/11/1815/11/18
Internet address

Keywords

  • Behavioral monitoring
  • Deep learning
  • Health analytics
  • Mental health
  • Social media
  • Visual features

Cite this

Nguyen, H., Nguyen, V., Nguyen, T., Larsen, M. E., O’Dea, B., Nguyen, D. T., ... Christensen, H. (2018). Jointly predicting affective and mental health scores using deep neural networks of visual cues on the web. In H. Hacid, W. Cellary, H. Wang, H-Y. Paik, & R. Zhou (Eds.), Web Information Systems Engineering – WISE 2018: 19th International Conference Dubai, United Arab Emirates, November 12–15, 2018 Proceedings, Part II (pp. 100-110). (Lecture Notes in Computer Science; Vol. 11234 ). Cham Switzerland: Springer. https://doi.org/10.1007/978-3-030-02925-8_7
Nguyen, Hung ; Nguyen, Van ; Nguyen, Thin ; Larsen, Mark E. ; O’Dea, Bridianne ; Nguyen, Duc Thanh ; Le, Trung ; Phung, Dinh ; Venkatesh, Svetha ; Christensen, Helen. / Jointly predicting affective and mental health scores using deep neural networks of visual cues on the web. Web Information Systems Engineering – WISE 2018: 19th International Conference Dubai, United Arab Emirates, November 12–15, 2018 Proceedings, Part II. editor / Hakim Hacid ; Wojciech Cellary ; Hua Wang ; Hye-Young Paik ; Rui Zhou. Cham Switzerland : Springer, 2018. pp. 100-110 (Lecture Notes in Computer Science).
@inproceedings{175fb1d3bf354fa18bd036faad9a87cc,
title = "Jointly predicting affective and mental health scores using deep neural networks of visual cues on the web",
abstract = "Despite the range of studies examining the relationship between mental health and social media data, not all prior studies have validated the social media markers against “ground truth”, or validated psychiatric information, in general community samples. Instead, researchers have approximated psychiatric diagnosis using user statements such as “I have been diagnosed as X”. Without “ground truth”, the value of predictive algorithms is highly questionable and potentially harmful. In addition, for social media data, whilst linguistic features have been widely identified as strong markers of mental health disorders, little is known about non-textual features on their links with the disorders. The current work is a longitudinal study during which participants’ mental health data, consisting of depression and anxiety scores, were collected fortnightly with a validated, diagnostic, clinical measure. Also, datasets with labels relevant to mental health scores, such as emotional scores, are also employed to improve the performance in prediction of mental health scores. This work introduces a deep neural network-based method integrating sub-networks on predicting affective scores and mental health outcomes from images. Experimental results have shown that in the both predictions of emotion and mental health scores, (1) deep features majorly outperform handcrafted ones and (2) the proposed network achieves better performance compared with separate networks.",
keywords = "Behavioral monitoring, Deep learning, Health analytics, Mental health, Social media, Visual features",
author = "Hung Nguyen and Van Nguyen and Thin Nguyen and Larsen, {Mark E.} and Bridianne O’Dea and Nguyen, {Duc Thanh} and Trung Le and Dinh Phung and Svetha Venkatesh and Helen Christensen",
year = "2018",
doi = "10.1007/978-3-030-02925-8_7",
language = "English",
isbn = "9783030029241",
series = "Lecture Notes in Computer Science",
publisher = "Springer",
pages = "100--110",
editor = "Hakim Hacid and Wojciech Cellary and Hua Wang and Hye-Young Paik and Rui Zhou",
booktitle = "Web Information Systems Engineering – WISE 2018",

}

Nguyen, H, Nguyen, V, Nguyen, T, Larsen, ME, O’Dea, B, Nguyen, DT, Le, T, Phung, D, Venkatesh, S & Christensen, H 2018, Jointly predicting affective and mental health scores using deep neural networks of visual cues on the web. in H Hacid, W Cellary, H Wang, H-Y Paik & R Zhou (eds), Web Information Systems Engineering – WISE 2018: 19th International Conference Dubai, United Arab Emirates, November 12–15, 2018 Proceedings, Part II. Lecture Notes in Computer Science, vol. 11234 , Springer, Cham Switzerland, pp. 100-110, International Conference on Web Information Systems Engineering 2018, Dubai, United Arab Emirates, 12/11/18. https://doi.org/10.1007/978-3-030-02925-8_7

Jointly predicting affective and mental health scores using deep neural networks of visual cues on the web. / Nguyen, Hung; Nguyen, Van; Nguyen, Thin; Larsen, Mark E.; O’Dea, Bridianne; Nguyen, Duc Thanh; Le, Trung; Phung, Dinh; Venkatesh, Svetha; Christensen, Helen.

Web Information Systems Engineering – WISE 2018: 19th International Conference Dubai, United Arab Emirates, November 12–15, 2018 Proceedings, Part II. ed. / Hakim Hacid; Wojciech Cellary; Hua Wang; Hye-Young Paik; Rui Zhou. Cham Switzerland : Springer, 2018. p. 100-110 (Lecture Notes in Computer Science; Vol. 11234 ).

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

TY - GEN

T1 - Jointly predicting affective and mental health scores using deep neural networks of visual cues on the web

AU - Nguyen, Hung

AU - Nguyen, Van

AU - Nguyen, Thin

AU - Larsen, Mark E.

AU - O’Dea, Bridianne

AU - Nguyen, Duc Thanh

AU - Le, Trung

AU - Phung, Dinh

AU - Venkatesh, Svetha

AU - Christensen, Helen

PY - 2018

Y1 - 2018

N2 - Despite the range of studies examining the relationship between mental health and social media data, not all prior studies have validated the social media markers against “ground truth”, or validated psychiatric information, in general community samples. Instead, researchers have approximated psychiatric diagnosis using user statements such as “I have been diagnosed as X”. Without “ground truth”, the value of predictive algorithms is highly questionable and potentially harmful. In addition, for social media data, whilst linguistic features have been widely identified as strong markers of mental health disorders, little is known about non-textual features on their links with the disorders. The current work is a longitudinal study during which participants’ mental health data, consisting of depression and anxiety scores, were collected fortnightly with a validated, diagnostic, clinical measure. Also, datasets with labels relevant to mental health scores, such as emotional scores, are also employed to improve the performance in prediction of mental health scores. This work introduces a deep neural network-based method integrating sub-networks on predicting affective scores and mental health outcomes from images. Experimental results have shown that in the both predictions of emotion and mental health scores, (1) deep features majorly outperform handcrafted ones and (2) the proposed network achieves better performance compared with separate networks.

AB - Despite the range of studies examining the relationship between mental health and social media data, not all prior studies have validated the social media markers against “ground truth”, or validated psychiatric information, in general community samples. Instead, researchers have approximated psychiatric diagnosis using user statements such as “I have been diagnosed as X”. Without “ground truth”, the value of predictive algorithms is highly questionable and potentially harmful. In addition, for social media data, whilst linguistic features have been widely identified as strong markers of mental health disorders, little is known about non-textual features on their links with the disorders. The current work is a longitudinal study during which participants’ mental health data, consisting of depression and anxiety scores, were collected fortnightly with a validated, diagnostic, clinical measure. Also, datasets with labels relevant to mental health scores, such as emotional scores, are also employed to improve the performance in prediction of mental health scores. This work introduces a deep neural network-based method integrating sub-networks on predicting affective scores and mental health outcomes from images. Experimental results have shown that in the both predictions of emotion and mental health scores, (1) deep features majorly outperform handcrafted ones and (2) the proposed network achieves better performance compared with separate networks.

KW - Behavioral monitoring

KW - Deep learning

KW - Health analytics

KW - Mental health

KW - Social media

KW - Visual features

UR - http://www.scopus.com/inward/record.url?scp=85055951111&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-02925-8_7

DO - 10.1007/978-3-030-02925-8_7

M3 - Conference Paper

SN - 9783030029241

T3 - Lecture Notes in Computer Science

SP - 100

EP - 110

BT - Web Information Systems Engineering – WISE 2018

A2 - Hacid, Hakim

A2 - Cellary, Wojciech

A2 - Wang, Hua

A2 - Paik, Hye-Young

A2 - Zhou, Rui

PB - Springer

CY - Cham Switzerland

ER -

Nguyen H, Nguyen V, Nguyen T, Larsen ME, O’Dea B, Nguyen DT et al. Jointly predicting affective and mental health scores using deep neural networks of visual cues on the web. In Hacid H, Cellary W, Wang H, Paik H-Y, Zhou R, editors, Web Information Systems Engineering – WISE 2018: 19th International Conference Dubai, United Arab Emirates, November 12–15, 2018 Proceedings, Part II. Cham Switzerland: Springer. 2018. p. 100-110. (Lecture Notes in Computer Science). https://doi.org/10.1007/978-3-030-02925-8_7