Hierarchical Neural Architecture Search for deep stereo matching

Xuelian Cheng, Yiran Zhong, Mehrtash Harandi, Yuchao Dai, Xiaojun Chang, Tom Drummond, Hongdong Li, Zongyuan Ge

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

188 Citations (Scopus)


To reduce the human efforts in neural network design, Neural Architecture Search (NAS) has been applied with remarkable success to various high-level vision tasks such as classification and semantic segmentation. The underlying idea for the NAS algorithm is straightforward, namely, to enable the network the ability to choose among a set of operations (e.g.,convolution with different filter sizes), one is able to find an optimal architecture that is better adapted to the problem at hand. However, so far the success of NAS has not been enjoyed by low-level geometric vision tasks such as stereo matching. This is partly due to the fact that state-of-the-art deep stereo matching networks, designed by humans, are already sheer in size. Directly applying the NAS to such massive structures is computationally prohibitive based on the currently available mainstream computing resources. In this paper, we propose the first end-to-end hierarchical NAS framework for deep stereo matching by incorporating task-specific human knowledge into the neural architecture search framework. Specifically, following the gold standard pipeline for deep stereo matching (ie., feature extraction – feature volume construction and dense matching), we optimize the architectures of the entire pipeline jointly. Extensive experiments show that our searched network outperforms all state-of-the-art deep stereo matching architectures and is ranked at the top 1 accuracy on KITTI stereo 2012, 2015 and Middlebury benchmarks, as well as the top 1 on SceneFlow dataset with a substantial improvement on the size of the network and the speed of inference. The code is available at LEAStereo.

Original languageEnglish
Title of host publicationAdvances in Neural Information Processing Systems 33 (NeurIPS 2020)
EditorsH. Lorochelle, M. Ranzato, R. Hadsell, M.F. Balcan, H. Lin
Place of PublicationSan Diego CA USA
PublisherNeural Information Processing Systems (NIPS)
Number of pages12
ISBN (Electronic)9781713829546
Publication statusPublished - 2020
EventAdvances in Neural Information Processing Systems 2020 - Virtual, Online, United States of America
Duration: 6 Dec 202012 Dec 2020
Conference number: 34th
https://proceedings.neurips.cc/paper/2020 (Proceedings )
https://nips.cc/Conferences/2020 (Website)

Publication series

NameAdvances in Neural Information Processing Systems
PublisherMorgan Kaufmann Publishers
ISSN (Print)1049-5258


ConferenceAdvances in Neural Information Processing Systems 2020
Abbreviated titleNeurIPS 2020
Country/TerritoryUnited States of America
CityVirtual, Online
Internet address

Cite this