Towards a robust differentiable architecture search under label noise

Christian Simon, Piotr Koniusz, Lars Petersson, Yan Han, Mehrtash Harandi

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

Abstract

Neural Architecture Search (NAS) is the game changer in designing robust neural architectures. Architectures designed by NAS outperform or compete with the best manual network designs in terms of accuracy, size, memory footprint and FLOPs. That said, previous studies focus on developing NAS algorithms for clean high quality data, a restrictive and somewhat unrealistic assumption. In this paper, focusing on the differentiable NAS algorithms, we show that vanilla NAS algorithms suffer from a performance loss if class labels are noisy. To combat this issue, we make use of the principle of information bottleneck as a regularizer. This leads us to develop a noise injecting operation that is included during the learning process, preventing the network from learning from noisy samples. Our empirical evaluations show that the noise injecting operation does not degrade the performance of the NAS algorithm if the data is indeed clean. In contrast, if the data is noisy, the architecture learned by our algorithm comfortably outperforms algorithms specifically equipped with sophisticated mechanisms to learn in the presence of label noise. In contrast to many algorithms designed to work in the presence of noisy labels, prior knowledge about the properties of the noise and its characteristics are not required for our algorithm.

Original languageEnglish
Title of host publicationProceedings, 2022 IEEE Winter Conference on Applications of Computer Vision, WACV 2022
EditorsEric Mortensen
Place of PublicationPiscataway NJ USA
PublisherIEEE, Institute of Electrical and Electronics Engineers
Pages3584-3594
Number of pages11
ISBN (Electronic)9781665409155
ISBN (Print)9781665409162
DOIs
Publication statusPublished - 2022
EventIEEE Winter Conference on Applications of Computer Vision 2021 - Online, United States of America
Duration: 4 Jan 20228 Jan 2022
https://wacv2022.thecvf.com/home (Website)
https://ieeexplore.ieee.org/xpl/conhome/9706406/proceeding (Proceedings)

Publication series

NameProceedings - 2022 IEEE/CVF Winter Conference on Applications of Computer Vision, WACV 2022
PublisherIEEE, Institute of Electrical and Electronics Engineers
ISSN (Print)2642-9381
ISSN (Electronic)2642-9381

Conference

ConferenceIEEE Winter Conference on Applications of Computer Vision 2021
Abbreviated titleWACV 2021
Country/TerritoryUnited States of America
Period4/01/228/01/22
Internet address

Keywords

  • Learning and Optimization Deep Learning
  • Statistical Methods

Cite this