Overcoming data limitation in medical Visual Question Answering

Binh D. Nguyen, Thanh-Toan Do, Binh X. Nguyen, Tuong Do, Erman Tjiputra, Quang D. Tran

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

72 Citations (Scopus)


Traditional approaches for Visual Question Answering (VQA) require large amount of labeled data for training. Unfortunately, such large scale data is usually not available for medical domain. In this paper, we propose a novel medical VQA framework that overcomes the labeled data limitation. The proposed framework explores the use of the unsupervised Denoising Auto-Encoder (DAE) and the supervised Meta-Learning. The advantage of DAE is to leverage the large amount of unlabeled images while the advantage of Meta-Learning is to learn meta-weights that quickly adapt to VQA problem with limited labeled data. By leveraging the advantages of these techniques, it allows the proposed framework to be efficiently trained using a small labeled training set. The experimental results show that our proposed method significantly outperforms the state-of-the-art medical VQA. The source code is available at https://github.com/aioz-ai/MICCAI19-MedVQA.

Original languageEnglish
Title of host publicationMedical Image Computing and Computer Assisted Intervention – MICCAI 2019
Subtitle of host publication22nd International Conference Shenzhen, China, October 13–17, 2019 Proceedings, Part IV
EditorsDinggang Shen, Tianming Liu, Terry M. Peters, Lawrence H. Staib, Caroline Essert, Sean Zhou, Pew-Thian Yap, Ali Khan
Place of PublicationCham Switzerland
Number of pages9
ISBN (Electronic)9783030322519
ISBN (Print)9783030322502
Publication statusPublished - 2019
Externally publishedYes
EventMedical Image Computing and Computer-Assisted Intervention 2019 - Shenzhen, China
Duration: 13 Oct 201917 Oct 2019
Conference number: 22nd
https://link.springer.com/book/10.1007/978-3-030-32239-7 (Proceedings)

Publication series

NameLecture Notes in Computer Science
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


ConferenceMedical Image Computing and Computer-Assisted Intervention 2019
Abbreviated titleMICCAI 2019
Internet address


  • Auto-Encoder
  • Meta-Learning
  • Visual Question Answering

Cite this