A Survey on Deep Learning and Explainability for Automatic Report Generation from Medical Images

Pablo Messina; Pablo Pino; Denis Parra; Alvaro Soto; Cecilia Besa; Sergio A. Uribe; Marcelo Andia; Cristián Tejos; Claudia Prieto; Daniel Capurro

doi:10.1145/3522747

A Survey on Deep Learning and Explainability for Automatic Report Generation from Medical Images

Pablo Messina, Pablo Pino, Denis Parra, Alvaro Soto, Cecilia Besa, Sergio A. Uribe, Marcelo Andia, Cristián Tejos, Claudia Prieto, Daniel Capurro

Research output: Contribution to journal › Article › Research › peer-review

Abstract

Every year physicians face an increasing demand of image-based diagnosis from patients, a problem that can be addressed with recent artificial intelligence methods. In this context, we survey works in the area of automatic report generation from medical images, with emphasis on methods using deep neural networks, with respect to (1) Datasets, (2) Architecture Design, (3) Explainability, and (4) Evaluation Metrics. Our survey identifies interesting developments but also remaining challenges. Among them, the current evaluation of generated reports is especially weak, since it mostly relies on traditional Natural Language Processing (NLP) metrics, which do not accurately capture medical correctness.

Original language	English
Article number	203
Number of pages	40
Journal	ACM Computing Surveys
Volume	54
Issue number	10S
DOIs	https://doi.org/10.1145/3522747
Publication status	Published - Sep 2022
Externally published	Yes

Access to Document

10.1145/3522747

Cite this

@article{3887ab6053a04b1cb5dbbad0682b9fa1,

title = "A Survey on Deep Learning and Explainability for Automatic Report Generation from Medical Images",

abstract = "Every year physicians face an increasing demand of image-based diagnosis from patients, a problem that can be addressed with recent artificial intelligence methods. In this context, we survey works in the area of automatic report generation from medical images, with emphasis on methods using deep neural networks, with respect to (1) Datasets, (2) Architecture Design, (3) Explainability, and (4) Evaluation Metrics. Our survey identifies interesting developments but also remaining challenges. Among them, the current evaluation of generated reports is especially weak, since it mostly relies on traditional Natural Language Processing (NLP) metrics, which do not accurately capture medical correctness.",

author = "Pablo Messina and Pablo Pino and Denis Parra and Alvaro Soto and Cecilia Besa and Uribe, {Sergio A.} and Marcelo Andia and Cristi{\'a}n Tejos and Claudia Prieto and Daniel Capurro",

year = "2022",

month = sep,

doi = "10.1145/3522747",

language = "English",

volume = "54",

journal = "ACM Computing Surveys",

issn = "0360-0300",

publisher = "Association for Computing Machinery (ACM)",

number = "10S",

}

TY - JOUR

T1 - A Survey on Deep Learning and Explainability for Automatic Report Generation from Medical Images

AU - Messina, Pablo

AU - Pino, Pablo

AU - Parra, Denis

AU - Soto, Alvaro

AU - Besa, Cecilia

AU - Uribe, Sergio A.

AU - Andia, Marcelo

AU - Tejos, Cristián

AU - Prieto, Claudia

AU - Capurro, Daniel

PY - 2022/9

Y1 - 2022/9

N2 - Every year physicians face an increasing demand of image-based diagnosis from patients, a problem that can be addressed with recent artificial intelligence methods. In this context, we survey works in the area of automatic report generation from medical images, with emphasis on methods using deep neural networks, with respect to (1) Datasets, (2) Architecture Design, (3) Explainability, and (4) Evaluation Metrics. Our survey identifies interesting developments but also remaining challenges. Among them, the current evaluation of generated reports is especially weak, since it mostly relies on traditional Natural Language Processing (NLP) metrics, which do not accurately capture medical correctness.

AB - Every year physicians face an increasing demand of image-based diagnosis from patients, a problem that can be addressed with recent artificial intelligence methods. In this context, we survey works in the area of automatic report generation from medical images, with emphasis on methods using deep neural networks, with respect to (1) Datasets, (2) Architecture Design, (3) Explainability, and (4) Evaluation Metrics. Our survey identifies interesting developments but also remaining challenges. Among them, the current evaluation of generated reports is especially weak, since it mostly relies on traditional Natural Language Processing (NLP) metrics, which do not accurately capture medical correctness.

U2 - 10.1145/3522747

DO - 10.1145/3522747

M3 - Article

VL - 54

JO - ACM Computing Surveys

JF - ACM Computing Surveys

SN - 0360-0300

IS - 10S

M1 - 203

ER -