Improving multi-label learning with missing labels by structured semantic correlations

Hao Yang, Joey Tianyi Zhou, Jianfei Cai

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

23 Citations (Scopus)


Multi-label learning has attracted significant interests in computer vision recently, finding applications in many vision tasks such as multiple object recognition and automatic image annotation. Associating multiple labels to a complex image is very difficult, not only due to the intricacy of describing the image, but also because of the incompleteness nature of the observed labels. Existing works on the problem either ignore the label-label and instance-instance correlations or just assume these correlations are linear and unstructured. Considering that semantic correlations between images are actually structured, in this paper we propose to incorporate structured semantic correlations to solve the missing label problem of multi-label learning. Specifically, we project images to the semantic space with an effective semantic descriptor. A semantic graph is then constructed on these images to capture the structured correlations between them. We utilize the semantic graph Laplacian as a smooth term in the multi-label learning formulation to incorporate the structured semantic correlations. Experimental results demonstrate the effectiveness of the proposed semantic descriptor and the usefulness of incorporating the structured semantic correlations. We achieve better results than state-of-the-art multi-label learning methods on four benchmark datasets.

Original languageEnglish
Title of host publicationComputer Vision – ECCV 2016
Subtitle of host publication14th European Conference Amsterdam, The Netherlands, October 11–14, 2016 Proceedings, Part I
EditorsBastian Leibe, Jiri Matas, Nicu Sebe, Max Welling
Place of PublicationCham Switzerland
Number of pages17
ISBN (Electronic)9783319464480
ISBN (Print)9783319464473
Publication statusPublished - 2016
Externally publishedYes
EventEuropean Conference on Computer Vision 2016 - Amsterdam, Netherlands
Duration: 11 Oct 201614 Oct 2016
Conference number: 14th (Proceedings)

Publication series

NameLecture Notes in Computer Science
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


ConferenceEuropean Conference on Computer Vision 2016
Abbreviated titleECCV 2016
Internet address

Cite this