Stereo matching by self-supervision of multiscopic vision

Weihao Yuan, Yazhan Zhang, Bingkun Wu, Siyu Zhu, Ping Tan, Michael Yu Wang, Qifeng Chen

Research output: Chapter in Book/Report/Conference proceedingConference PaperOther

2 Citations (Scopus)


Self-supervised learning for depth estimation possesses several advantages over supervised learning. The benefits of no need for ground-truth depth, online fine-tuning, and better generalization with unlimited data attract researchers to seek self-supervised solutions. In this work, we propose a new self-supervised framework for stereo matching utilizing multiple images captured at aligned camera positions. A cross photometric loss, an uncertainty-aware mutual-supervision loss, and a new smoothness loss are introduced to optimize the network in learning disparity maps end-to-end without ground-truth depth information. To train this framework, we build a new multiscopic dataset consisting of synthetic images rendered by 3D engines and real images captured by real cameras. After being trained with only the synthetic images, our network can perform well in unseen outdoor scenes. Our experiment shows that our model obtains better disparity maps than previous unsupervised methods on the KITTI dataset and is comparable to supervised methods when generalized to unseen data. Our source code and dataset are available at

Original languageEnglish
Title of host publicationIEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2021
PublisherIEEE, Institute of Electrical and Electronics Engineers
Number of pages8
ISBN (Electronic)9781665417143
Publication statusPublished - 2021
Externally publishedYes
EventIEEE/RSJ International Conference on Intelligent Robots and Systems 2021 - Online, Prague, Czech Republic
Duration: 27 Sep 20211 Oct 2021 (Proceedings)

Publication series

NameIEEE International Conference on Intelligent Robots and Systems
ISSN (Print)2153-0858
ISSN (Electronic)2153-0866


ConferenceIEEE/RSJ International Conference on Intelligent Robots and Systems 2021
Abbreviated titleIROS 2021
Country/TerritoryCzech Republic
Internet address

Cite this