One-Shot Neural architecture search via novelty driven sampling

Miao Zhang, Huiqi Li, Shirui Pan, Taoping Liu, Steven Su

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

Abstract

One-Shot Neural architecture search (NAS) has received wide attentions due to its computational efficiency. Most state-of-the-art One-Shot NAS methods use the validation accuracy based on inheriting weights from the supernet as the stepping stone to search for the best performing architecture, adopting a bilevel optimization pattern with assuming this validation accuracy approximates to the test accuracy after re-training. However, recent works have found that there is no positive correlation between the above validation accuracy and test accuracy for these One-Shot NAS methods, and this reward based sampling for supernet training also entails the rich-get-richer problem. To handle this deceptive problem, this paper presents a new approach, Efficient Novelty-driven Neural Architecture Search, to sample the most abnormal architecture to train the supernet. Specifically, a single-path supernet is adopted, and only the weights of a single architecture sampled by our novelty search are optimized in each step to reduce the memory demand greatly. Experiments demonstrate the effectiveness and efficiency of our novelty search based architecture sampling method.
Original languageEnglish
Title of host publicationProceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence
EditorsChristian Bessiere
Place of PublicationMarina del Rey CA USA
PublisherAssociation for the Advancement of Artificial Intelligence (AAAI)
Pages3188-3194
Number of pages7
ISBN (Electronic)9780999241165
DOIs
Publication statusPublished - 2020
EventInternational Joint Conference on Artificial Intelligence-Pacific Rim International Conference on Artificial Intelligence 2020 - Yokohama, Japan
Duration: 7 Jan 202115 Jan 2021
Conference number: 29th/17th
https://www.ijcai.org/Proceedings/2020/ (Proceedings)
https://ijcai20.org (Website)

Conference

ConferenceInternational Joint Conference on Artificial Intelligence-Pacific Rim International Conference on Artificial Intelligence 2020
Abbreviated titleIJCAI-PRICAI 2020
CountryJapan
CityYokohama
Period7/01/2115/01/21
OtherIJCAI-PRICAI 2020, the 29th International Joint Conference on Artificial Intelligence and the 17th Pacific Rim International Conference on Artificial Intelligence!IJCAI-PRICAI2020 will take place January 7-15, 2021 online in a virtual reality in Japanese Standard Time (JST) zone.
Internet address

Keywords

  • Machine Learning
  • Deep Learning
  • Heuristic Search and Game Playing
  • Heuristic Search and Machine Learning
  • Meta-Reasoning and Meta-heuristics
  • Online Learning

Cite this