Deep learning for pollen allergy surveillance from twitter in Australia

Jia Rong, Sandra Michalska, Sudha Subramani, Jiahua Du, Hua Wang

Research output: Contribution to journalArticleResearchpeer-review

5 Citations (Scopus)

Abstract

Background: The paper introduces a deep learning-based approach for real-time detection and insights generation about one of the most prevalent chronic conditions in Australia - Pollen allergy. The popular social media platform is used for data collection as cost-effective and unobtrusive alternative for public health monitoring to complement the traditional survey-based approaches. 

Methods: The data was extracted from Twitter based on pre-defined keywords (i.e. 'hayfever' OR 'hay fever') throughout the period of 6 months, covering the high pollen season in Australia. The following deep learning architectures were adopted in the experiments: CNN, RNN, LSTM and GRU. Both default (GloVe) and domain-specific (HF) word embeddings were used in training the classifiers. Standard evaluation metrics (i.e. Accuracy, Precision and Recall) were calculated for the results validation. Finally, visual correlation with weather variables was performed. 

Results: The neural networks-based approach was able to correctly identify the implicit mentions of the symptoms and treatments, even unseen previously (accuracy up to 87.9% for GRU with GloVe embeddings of 300 dimensions). 

Conclusions: The system addresses the shortcomings of the conventional machine learning techniques with manual feature-engineering that prove limiting when exposed to a wide range of non-standard expressions relating to medical concepts. The case-study presented demonstrates an application of 'black-box' approach to the real-world problem, along with its internal workings demonstration towards more transparent, interpretable and reproducible decision-making in health informatics domain.

Original languageEnglish
Article number208
Number of pages13
JournalBMC Medical Informatics and Decision Making
Volume19
Issue number1
DOIs
Publication statusPublished - 8 Nov 2019

Keywords

  • Deep learning
  • Hay fever
  • Pollen allergy
  • Twitter mining

Cite this