Exploiting local and generic features for accurate skin lesions classification using clinical and dermoscopy imaging

Zongyuan Ge, Sergey Demyanov, Behzad Bozorgtabar, Mani Abedini, Rajib Chakravorty, Adrian Bowling, Rahil Garnavi

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

47 Citations (Scopus)


Similarity in appearance between various skin diseases, often makes it challenging for clinicians to identify the type of skin condition, and the accuracy is highly reliant on the level of expertise. There is also a great degree of subjectivity and inter/intra observer variability found in the clinical practices. In this paper, we propose a method for automatic skin diseases recognition that combines two different types of deep convolutional neural network features. We hold the hypothesis that it is equally important to capture global features such as color and lesion shape, as well as local features such as local patterns within the lesion area. The proposed method leverages deep residual network to represent global information, and bilinear pooling technique which allows to extract local features to differentiate between skin conditions with subtle visual differences in local regions. We have evaluated our proposed method on MoleMap dataset with 32,195 and ISBI-2016 challenge dataset with 1,279 skin images. Without any lesion localisation or segmentation, our proposed method has achieved state-of-the-art results on the large-scale MoleMap datasets with 15 various disease categories and multiple imaging modalities, and compares favorably with the best method on ISBI-2016 Melanoma challenge dataset.

Original languageEnglish
Title of host publication2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017)
EditorsSimon Warfield, Arrete Munoz-Barrutia
Place of PublicationPiscataway NJ USA
PublisherIEEE, Institute of Electrical and Electronics Engineers
Number of pages5
ISBN (Electronic)9781509011728
ISBN (Print)9781509011735
Publication statusPublished - 15 Jun 2017
Externally publishedYes
EventIEEE International Symposium on Biomedical Imaging (ISBI) 2017 - Melbourne Convention and Exhibition Centre, Melbourne, Australia
Duration: 18 Apr 201721 Apr 2017
Conference number: 14th


ConferenceIEEE International Symposium on Biomedical Imaging (ISBI) 2017
Abbreviated titleISBI 2017
OtherISBI is a joint initiative from the IEEE Signal Processing Society (SPS) and the IEEE Engineering in Medicine and Biology Society (EMBS). The 2017 meeting will include tutorials, and a scientific program composed of plenary talks, invited special sessions, challenges, as well as oral and poster presentations of peer-reviewed papers.
Internet address


  • Bilinear pooling
  • Deep convolutional neural network (DCNN)
  • Feature fusion
  • Skin disease recognition

Cite this