Abstract
One-Shot Neural architecture search (NAS) has received wide attentions due to its computational efficiency. Most state-of-the-art One-Shot NAS methods use the validation accuracy based on inheriting weights from the supernet as the stepping stone to search for the best performing architecture, adopting a bilevel optimization pattern with assuming this validation accuracy approximates to the test accuracy after re-training. However, recent works have found that there is no positive correlation between the above validation accuracy and test accuracy for these One-Shot NAS methods, and this reward based sampling for supernet training also entails the rich-get-richer problem. To handle this deceptive problem, this paper presents a new approach, Efficient Novelty-driven Neural Architecture Search, to sample the most abnormal architecture to train the supernet. Specifically, a single-path supernet is adopted, and only the weights of a single architecture sampled by our novelty search are optimized in each step to reduce the memory demand greatly. Experiments demonstrate the effectiveness and efficiency of our novelty search based architecture sampling method.
Original language | English |
---|---|
Title of host publication | Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence |
Editors | Christian Bessiere |
Place of Publication | Marina del Rey CA USA |
Publisher | Association for the Advancement of Artificial Intelligence (AAAI) |
Pages | 3188-3194 |
Number of pages | 7 |
ISBN (Electronic) | 9780999241165 |
DOIs | |
Publication status | Published - 2020 |
Event | International Joint Conference on Artificial Intelligence-Pacific Rim International Conference on Artificial Intelligence 2020 - Yokohama, Japan Duration: 7 Jan 2021 → 15 Jan 2021 Conference number: 29th/17th https://www.ijcai.org/Proceedings/2020/ (Proceedings) https://ijcai20.org (Website) |
Conference
Conference | International Joint Conference on Artificial Intelligence-Pacific Rim International Conference on Artificial Intelligence 2020 |
---|---|
Abbreviated title | IJCAI-PRICAI 2020 |
Country/Territory | Japan |
City | Yokohama |
Period | 7/01/21 → 15/01/21 |
Other | IJCAI-PRICAI 2020, the 29th International Joint Conference on Artificial Intelligence and the 17th Pacific Rim International Conference on Artificial Intelligence!IJCAI-PRICAI2020 will take place January 7-15, 2021 online in a virtual reality in Japanese Standard Time (JST) zone. |
Internet address |
|
Keywords
- Machine Learning
- Deep Learning
- Heuristic Search and Game Playing
- Heuristic Search and Machine Learning
- Meta-Reasoning and Meta-heuristics
- Online Learning