Improving deep lesion detection using 3D contextual and spatial attention

Qingyi Tao, Zongyuan Ge, Jianfei Cai, Jianxiong Yin, Simon See

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

18 Citations (Scopus)


Lesion detection from computed tomography (CT) scans is challenging compared to natural object detection because of two major reasons: small lesion size and small inter-class variation. Firstly, the lesions usually only occupy a small region in the CT image. The feature of such small region may not be able to provide sufficient information due to its limited spatial feature resolution. Secondly, in CT scans, the lesions are often indistinguishable from the background since the lesion and non-lesion areas may have very similar appearances. To tackle both problems, we need to enrich the feature representation and improve the feature discriminativeness. Therefore, we introduce a dual-attention mechanism to the 3D contextual lesion detection framework, including the cross-slice contextual attention to selectively aggregate the information from different slices through a soft re-sampling process. Moreover, we propose intra-slice spatial attention to focus the feature learning in the most prominent regions. Our method can be easily trained end-to-end without adding heavy overhead on the base detection network. We use DeepLesion dataset and train a universal lesion detector to detect all kinds of lesions such as liver tumors, lung nodules, and so on. The results show that our model can significantly boost the results of the baseline lesion detector (with 3D contextual information) but using much fewer slices.

Original languageEnglish
Title of host publicationMedical Image Computing and Computer Assisted Intervention – MICCAI 2019
Subtitle of host publication22nd International Conference Shenzhen, China, October 13–17, 2019 Proceedings, Part VI
EditorsDinggang Shen, Tianming Liu, Terry M. Peters, Lawrence H. Staib, Caroline Essert, Sean Zhou, Pew-Thian Yap, Ali Khan
Place of PublicationCham Switzerland
Number of pages9
ISBN (Electronic)9783030322267
ISBN (Print)9783030322250
Publication statusPublished - 2019
EventMedical Image Computing and Computer-Assisted Intervention 2019 - Shenzhen, China
Duration: 13 Oct 201917 Oct 2019
Conference number: 22nd (Proceedings)

Publication series

NameLecture Notes in Computer Science
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


ConferenceMedical Image Computing and Computer-Assisted Intervention 2019
Abbreviated titleMICCAI 2019
Internet address

Cite this