Bayesian generative active deep learning

Toan Tran, Thanh-Toan Do, Ian Reid, Gustavo Carneiro

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

1 Citation (Scopus)

Abstract

Deep learning models have demonstrated outstanding performance in several problems, but their training process tends to require immense amounts of computational and human resources for training and labeling, constraining the types of problems that can be tackled. Therefore, the design of effective training methods that require small labeled training sets is an important research direction that will allow a more effective use of resources. Among current approaches designed to address this issue, two are particularly interesting: data augmentation and active learning. Data augmentation achieves this goal by artificially generating new training points, while active learning relies on the selection of the "most informative" subset of unlabeled training samples to be labelled by an oracle. Although successful in practice, data augmentation can waste computational resources because it indiscriminately generates samples that are not guaranteed to be informative, and active learning selects a small subset of informative samples (from a large un-annotated set) that may be insufficient for the training process. In this paper, we propose a Bayesian generative active deep learning approach that combines active learning with data augmentation - we provide theoretical and empirical evidence (MNIST, CIFAR-{10,100}, and SVHN) that our approach has more efficient training and better classification results than data augmentation and active learning.

Original languageEnglish
Title of host publicationProceedings of the 36th International Conference on Machine Learning
EditorsKamalika Chaudhuri, Ruslan Salakhutdinov
Place of PublicationStroudsburg PA USA
PublisherInternational Machine Learning Society (IMLS)
Pages6295-6304
Number of pages10
ISBN (Electronic)9781510886988
Publication statusPublished - 2019
Externally publishedYes
EventInternational Conference on Machine Learning 2019 - Long Beach, United States of America
Duration: 9 Jun 201915 Jun 2019
Conference number: 36th
https://icml.cc/Conferences/2019 (Website)
http://proceedings.mlr.press/v97/ (Proceedings)

Conference

ConferenceInternational Conference on Machine Learning 2019
Abbreviated titleICML 2019
CountryUnited States of America
CityLong Beach
Period9/06/1915/06/19
Internet address

Cite this