TY - JOUR
T1 - Multi-modal mining of crowd-sourced data
T2 - Efficient provision of humanitarian aid to remote regions affected by natural disasters
AU - Khanmohammadi, Sadegh
AU - Golafshani, Emad
AU - Bai, Yu
AU - Li, Heng
AU - Bazli, Milad
AU - Arashpour, Mehrdad
N1 - Funding Information:
The authors are grateful for support from the Australian Research Council (ARC) through the Linkage project ( LP180101080 ). The authors acknowledge the contributions of the members of ASCII Lab at Monash University for critiquing the manuscript and providing constructive feedback.
Funding Information:
The authors are grateful for support from the Australian Research Council (ARC) through the Linkage project (LP180101080). The authors acknowledge the contributions of the members of ASCII Lab at Monash University for critiquing the manuscript and providing constructive feedback.
Publisher Copyright:
© 2023 The Authors
PY - 2023/10/1
Y1 - 2023/10/1
N2 - Data mining applications have the potential to address current deficiencies in the provision of humanitarian aid in natural disasters. Simultaneous text and image analysis in crowd-sourced data can improve the quality of humanitarian aid information. Specifically, we select Bidirectional Encoder Representations from Transformers (BERT) and its descendant ALBERT as pre-trained deep networks for the text modality, while we choose ConvNeXt, RegNet, and Faster RCNN for the image modality. The developed framework demonstrates its application in classifying humanitarian aid through three key aspects. Firstly, it illustrates the effective performance of ConvNeXt and BERT in the classification of humanitarian aid. Secondly, it investigates the efficiency of generative adversarial networks (GAN) in generating synthetic images for imbalanced input datasets. This approach improves the accuracy, precision, recall, and F1-score of the framework when applied to unseen test data. Finally, the study highlights the potential use of SHapley Additive exPlanations (SHAP) for interpreting the behaviour of the developed framework, supporting the timely classification of humanitarian aid information from crowd-sourced data after natural disasters.
AB - Data mining applications have the potential to address current deficiencies in the provision of humanitarian aid in natural disasters. Simultaneous text and image analysis in crowd-sourced data can improve the quality of humanitarian aid information. Specifically, we select Bidirectional Encoder Representations from Transformers (BERT) and its descendant ALBERT as pre-trained deep networks for the text modality, while we choose ConvNeXt, RegNet, and Faster RCNN for the image modality. The developed framework demonstrates its application in classifying humanitarian aid through three key aspects. Firstly, it illustrates the effective performance of ConvNeXt and BERT in the classification of humanitarian aid. Secondly, it investigates the efficiency of generative adversarial networks (GAN) in generating synthetic images for imbalanced input datasets. This approach improves the accuracy, precision, recall, and F1-score of the framework when applied to unseen test data. Finally, the study highlights the potential use of SHapley Additive exPlanations (SHAP) for interpreting the behaviour of the developed framework, supporting the timely classification of humanitarian aid information from crowd-sourced data after natural disasters.
KW - Artificial intelligence (AI)
KW - Data mining
KW - Deep neural networks (DNN)
KW - Generative adversarial networks (GAN)
KW - Humanitarian aid
KW - SHapley additive exPlanations (SHAP)
UR - http://www.scopus.com/inward/record.url?scp=85170026500&partnerID=8YFLogxK
U2 - 10.1016/j.ijdrr.2023.103972
DO - 10.1016/j.ijdrr.2023.103972
M3 - Article
AN - SCOPUS:85170026500
SN - 2212-4209
VL - 96
JO - International Journal of Disaster Risk Reduction
JF - International Journal of Disaster Risk Reduction
M1 - 103972
ER -