Deep multiscale convolutional feature learning for weakly supervised localization of chest pathologies in X-ray images

Suman Sedai, Dwarikanath Mahapatra, Zongyuan Ge, Rajib Chakravorty, Rahil Garnavi

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

Abstract

Localization of chest pathologies in chest X-ray images is a challenging task because of their varying sizes and appearances. We propose a novel weakly supervised method to localize chest pathologies using class aware deep multiscale feature learning. Our method leverages intermediate feature maps from CNN layers at different stages of a deep network during the training of a classification model using image level annotations of pathologies. During the training phase, a set of layer relevance weights are learned for each pathology class and the CNN is optimized to perform pathology classification by convex combination of feature maps from both shallow and deep layers using the learned weights. During the test phase, to localize the predicted pathology, the multiscale attention map is obtained by convex combination of class activation maps from each stage using the layer relevance weights learned during the training phase. We have validated our method using 112000 X-ray images and compared with the state-of-the-art localization methods. We experimentally demonstrate that the proposed weakly supervised method can improve the localization performance of small pathologies such as nodule and mass while giving comparable performance for bigger pathologies e.g., Cardiomegaly.

Original languageEnglish
Title of host publicationMachine Learning in Medical Imaging
Subtitle of host publication9th International Workshop, MLMI 2018, Held in Conjunction with MICCAI 2018, Granada, Spain, September 16, 2018, Proceedings
EditorsYinghuan Shi, Heung-Il Suk, Mingxia Liu
Place of PublicationCham Switzerland
PublisherSpringer
Pages267-275
Number of pages9
Volume11046
ISBN (Electronic)9783030009199
ISBN (Print)9783030009182
DOIs
Publication statusPublished - 1 Jan 2018
Externally publishedYes
EventInternational Workshop on Machine Learning in Medical Imaging (MLMI) 2018 - Granada Conference Centre, Granada, Spain
Duration: 16 Sep 201816 Sep 2018
Conference number: 9th
http://mlmi2018.web.unc.edu/

Publication series

NameLecture Notes in Computer Science
PublisherSpringer
Volume11046
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

ConferenceInternational Workshop on Machine Learning in Medical Imaging (MLMI) 2018
Abbreviated titleMLMI 2018
CountrySpain
CityGranada
Period16/09/1816/09/18
Internet address

Keywords

  • Weakly supervised learning
  • X-ray pathology classification

Cite this

Sedai, S., Mahapatra, D., Ge, Z., Chakravorty, R., & Garnavi, R. (2018). Deep multiscale convolutional feature learning for weakly supervised localization of chest pathologies in X-ray images. In Y. Shi, H-I. Suk, & M. Liu (Eds.), Machine Learning in Medical Imaging: 9th International Workshop, MLMI 2018, Held in Conjunction with MICCAI 2018, Granada, Spain, September 16, 2018, Proceedings (Vol. 11046, pp. 267-275). (Lecture Notes in Computer Science; Vol. 11046). Cham Switzerland: Springer. https://doi.org/10.1007/978-3-030-00919-9_31
Sedai, Suman ; Mahapatra, Dwarikanath ; Ge, Zongyuan ; Chakravorty, Rajib ; Garnavi, Rahil. / Deep multiscale convolutional feature learning for weakly supervised localization of chest pathologies in X-ray images. Machine Learning in Medical Imaging: 9th International Workshop, MLMI 2018, Held in Conjunction with MICCAI 2018, Granada, Spain, September 16, 2018, Proceedings. editor / Yinghuan Shi ; Heung-Il Suk ; Mingxia Liu. Vol. 11046 Cham Switzerland : Springer, 2018. pp. 267-275 (Lecture Notes in Computer Science).
@inproceedings{a903a493bd7d4cf1aed91ecaba1cdf94,
title = "Deep multiscale convolutional feature learning for weakly supervised localization of chest pathologies in X-ray images",
abstract = "Localization of chest pathologies in chest X-ray images is a challenging task because of their varying sizes and appearances. We propose a novel weakly supervised method to localize chest pathologies using class aware deep multiscale feature learning. Our method leverages intermediate feature maps from CNN layers at different stages of a deep network during the training of a classification model using image level annotations of pathologies. During the training phase, a set of layer relevance weights are learned for each pathology class and the CNN is optimized to perform pathology classification by convex combination of feature maps from both shallow and deep layers using the learned weights. During the test phase, to localize the predicted pathology, the multiscale attention map is obtained by convex combination of class activation maps from each stage using the layer relevance weights learned during the training phase. We have validated our method using 112000 X-ray images and compared with the state-of-the-art localization methods. We experimentally demonstrate that the proposed weakly supervised method can improve the localization performance of small pathologies such as nodule and mass while giving comparable performance for bigger pathologies e.g., Cardiomegaly.",
keywords = "Weakly supervised learning, X-ray pathology classification",
author = "Suman Sedai and Dwarikanath Mahapatra and Zongyuan Ge and Rajib Chakravorty and Rahil Garnavi",
year = "2018",
month = "1",
day = "1",
doi = "10.1007/978-3-030-00919-9_31",
language = "English",
isbn = "9783030009182",
volume = "11046",
series = "Lecture Notes in Computer Science",
publisher = "Springer",
pages = "267--275",
editor = "Yinghuan Shi and Heung-Il Suk and Mingxia Liu",
booktitle = "Machine Learning in Medical Imaging",

}

Sedai, S, Mahapatra, D, Ge, Z, Chakravorty, R & Garnavi, R 2018, Deep multiscale convolutional feature learning for weakly supervised localization of chest pathologies in X-ray images. in Y Shi, H-I Suk & M Liu (eds), Machine Learning in Medical Imaging: 9th International Workshop, MLMI 2018, Held in Conjunction with MICCAI 2018, Granada, Spain, September 16, 2018, Proceedings. vol. 11046, Lecture Notes in Computer Science, vol. 11046, Springer, Cham Switzerland, pp. 267-275, International Workshop on Machine Learning in Medical Imaging (MLMI) 2018, Granada, Spain, 16/09/18. https://doi.org/10.1007/978-3-030-00919-9_31

Deep multiscale convolutional feature learning for weakly supervised localization of chest pathologies in X-ray images. / Sedai, Suman; Mahapatra, Dwarikanath; Ge, Zongyuan; Chakravorty, Rajib; Garnavi, Rahil.

Machine Learning in Medical Imaging: 9th International Workshop, MLMI 2018, Held in Conjunction with MICCAI 2018, Granada, Spain, September 16, 2018, Proceedings. ed. / Yinghuan Shi; Heung-Il Suk; Mingxia Liu. Vol. 11046 Cham Switzerland : Springer, 2018. p. 267-275 (Lecture Notes in Computer Science; Vol. 11046).

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

TY - GEN

T1 - Deep multiscale convolutional feature learning for weakly supervised localization of chest pathologies in X-ray images

AU - Sedai, Suman

AU - Mahapatra, Dwarikanath

AU - Ge, Zongyuan

AU - Chakravorty, Rajib

AU - Garnavi, Rahil

PY - 2018/1/1

Y1 - 2018/1/1

N2 - Localization of chest pathologies in chest X-ray images is a challenging task because of their varying sizes and appearances. We propose a novel weakly supervised method to localize chest pathologies using class aware deep multiscale feature learning. Our method leverages intermediate feature maps from CNN layers at different stages of a deep network during the training of a classification model using image level annotations of pathologies. During the training phase, a set of layer relevance weights are learned for each pathology class and the CNN is optimized to perform pathology classification by convex combination of feature maps from both shallow and deep layers using the learned weights. During the test phase, to localize the predicted pathology, the multiscale attention map is obtained by convex combination of class activation maps from each stage using the layer relevance weights learned during the training phase. We have validated our method using 112000 X-ray images and compared with the state-of-the-art localization methods. We experimentally demonstrate that the proposed weakly supervised method can improve the localization performance of small pathologies such as nodule and mass while giving comparable performance for bigger pathologies e.g., Cardiomegaly.

AB - Localization of chest pathologies in chest X-ray images is a challenging task because of their varying sizes and appearances. We propose a novel weakly supervised method to localize chest pathologies using class aware deep multiscale feature learning. Our method leverages intermediate feature maps from CNN layers at different stages of a deep network during the training of a classification model using image level annotations of pathologies. During the training phase, a set of layer relevance weights are learned for each pathology class and the CNN is optimized to perform pathology classification by convex combination of feature maps from both shallow and deep layers using the learned weights. During the test phase, to localize the predicted pathology, the multiscale attention map is obtained by convex combination of class activation maps from each stage using the layer relevance weights learned during the training phase. We have validated our method using 112000 X-ray images and compared with the state-of-the-art localization methods. We experimentally demonstrate that the proposed weakly supervised method can improve the localization performance of small pathologies such as nodule and mass while giving comparable performance for bigger pathologies e.g., Cardiomegaly.

KW - Weakly supervised learning

KW - X-ray pathology classification

UR - http://www.scopus.com/inward/record.url?scp=85054545797&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-00919-9_31

DO - 10.1007/978-3-030-00919-9_31

M3 - Conference Paper

SN - 9783030009182

VL - 11046

T3 - Lecture Notes in Computer Science

SP - 267

EP - 275

BT - Machine Learning in Medical Imaging

A2 - Shi, Yinghuan

A2 - Suk, Heung-Il

A2 - Liu, Mingxia

PB - Springer

CY - Cham Switzerland

ER -

Sedai S, Mahapatra D, Ge Z, Chakravorty R, Garnavi R. Deep multiscale convolutional feature learning for weakly supervised localization of chest pathologies in X-ray images. In Shi Y, Suk H-I, Liu M, editors, Machine Learning in Medical Imaging: 9th International Workshop, MLMI 2018, Held in Conjunction with MICCAI 2018, Granada, Spain, September 16, 2018, Proceedings. Vol. 11046. Cham Switzerland: Springer. 2018. p. 267-275. (Lecture Notes in Computer Science). https://doi.org/10.1007/978-3-030-00919-9_31