A preliminary approach to semi-supervised learning in convolutional neural networks applying “sleep-wake” cycles

Mikel Elkano, Humberto Bustince, Andrew Paplinski

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

Abstract

The scarcity of labeled data has limited the capacity of convolutional neural networks (CNNs) until not long ago and still represents a serious problem in a number of image processing applications. Unsupervised methods have been shown to perform well in feature extraction and clustering tasks, but further investigation on unsupervised solutions for CNNs is needed. In this work, we propose a bio-inspired methodology that applies a deep generative model to help the CNN take advantage of unlabeled data and improve its classification performance. Inspired by the human “sleep-wake cycles”, the proposed method divides the learning process into sleep and waking periods. During the waking period, both the generative model and the CNN learn from real training data simultaneously. When sleep begins, none of the networks receive real data and the generative model creates a synthetic dataset from which the CNN learns. The experimental results showed that the generative model was able to teach the CNN and improve its classification performance.

Original languageEnglish
Title of host publicationNeural Information Processing
Subtitle of host publication24th International Conference, ICONIP 2017, Guangzhou, China, November 14-18, 2017, Proceedings, Part IV
EditorsDerong Liu, Shengli Xie, Yuanqing Li, Dongbin Zhao, El-Sayed M. El-Alfy
Place of PublicationCham Switzerland
PublisherSpringer
Pages466-474
Number of pages9
ISBN (Electronic)9783319700939
ISBN (Print)9783319700922
DOIs
Publication statusPublished - 2017
EventInternational Conference on Neural Information Processing 2017 - Guangzhou, China
Duration: 14 Nov 201718 Nov 2017
Conference number: 24th
https://link.springer.com/book/10.1007/978-3-319-70087-8 (Proceedings)

Publication series

NameLecture Notes in Computer Science
PublisherSpringer
Volume10637
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

ConferenceInternational Conference on Neural Information Processing 2017
Abbreviated titleICONIP 2017
CountryChina
CityGuangzhou
Period14/11/1718/11/17
Internet address

Keywords

  • Convolutional neural networks
  • Deep learning
  • Generative models
  • Semi-supervised learning
  • Sleep-wake cycles
  • Variational autoencoders

Cite this

Elkano, M., Bustince, H., & Paplinski, A. (2017). A preliminary approach to semi-supervised learning in convolutional neural networks applying “sleep-wake” cycles. In D. Liu, S. Xie, Y. Li, D. Zhao, & E-S. M. El-Alfy (Eds.), Neural Information Processing: 24th International Conference, ICONIP 2017, Guangzhou, China, November 14-18, 2017, Proceedings, Part IV (pp. 466-474). (Lecture Notes in Computer Science; Vol. 10637). Springer. https://doi.org/10.1007/978-3-319-70093-9_49