Teaching semi-supervised classifier via generalized distillation

Chen Gong, Xiaojun Chang, Meng Fang, Jian Yang

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

16 Citations (Scopus)


Semi-Supervised Learning (SSL) is able to build reliable classifier with very scarce labeled examples by properly utilizing the abundant unlabeled examples. However, existing SSL algorithms often yield unsatisfactory performance due to the lack of supervision information. To address this issue, this paper formulates SSL as a Generalized Distillation (GD) problem, which treats existing SSL algorithm as a learner and introduces a teacher to guide the learner's training process. Specifically, the intelligent teacher holds the privileged knowledge that “explains” the training data but remains unknown to the learner, and the teacher should convey its rich knowledge to the imperfect learner through a specific teaching function. After that, the learner gains knowledge by “imitating” the output of the teaching function under an optimization framework. Therefore, the learner in our algorithm learns from both the teacher and the training data, so its output can be substantially distilled and enhanced. By deriving the Rademacher complexity and error bounds of the proposed algorithm, the usefulness of the introduced teacher is theoretically demonstrated. The superiority of our algorithm to the state-of-the-art methods has also been demonstrated by the experiments on different datasets with various sources of privileged knowledge.

Original languageEnglish
Title of host publicationProceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI 2018
EditorsJerome Lang
Place of PublicationMarina del Rey CA USA
PublisherAssociation for the Advancement of Artificial Intelligence (AAAI)
Number of pages7
ISBN (Electronic)9780999241127
Publication statusPublished - 2018
Externally publishedYes
EventInternational Joint Conference on Artificial Intelligence 2018 - Stockholm, Sweden
Duration: 13 Jul 201819 Jul 2018
Conference number: 27th
https://www.ijcai.org/proceedings/2018/ (Proceedings)


ConferenceInternational Joint Conference on Artificial Intelligence 2018
Abbreviated titleIJCAI 2018
Internet address


  • Classification
  • Machine Learning
  • Semi-Supervised Learning

Cite this