Abstract
We present 'just-in-time reconstruction' as realtime image-guided inpainting of a map with arbitrary scale and sparsity to generate a fully dense depth map for the image. In particular, our goal is to inpaint a sparse map - obtained from either a monocular visual SLAM system or a sparse sensor - using a single-view depth prediction network as a virtual depth sensor. We adopt a fairly standard approach to data fusion, to produce a fused depth map by performing inference over a novel fully-connected Conditional Random Field (CRF) which is parameterized by the input depth maps and their pixel-wise confidence weights. Crucially, we obtain the confidence weights that parameterize the CRF model in a data-dependent manner via Convolutional Neural Networks (CNNs) which are trained to model the conditional depth error distributions given each source of input depth map and the associated RGB image. Our CRF model penalises absolute depth error in its nodes and pairwise scale-invariant depth error in its edges, and the confidence-based fusion minimizes the impact of outlier input depth values on the fused result. We demonstrate the flexibility of our method by real-time inpainting of ORB-SLAM, Kinect, and LIDAR depth maps acquired both indoors and outdoors at arbitrary scale and varied amount of irregular sparsity.
Original language | English |
---|---|
Title of host publication | 2018 IEEE International Conference on Robotics and Automation (ICRA 2018) |
Editors | Peter Corke, Nancy M. Amato, Megan Emmons, Yoshihiko Nakamura, Markus Vincze |
Place of Publication | Piscataway NJ USA |
Publisher | IEEE, Institute of Electrical and Electronics Engineers |
Pages | 4977-4984 |
Number of pages | 8 |
ISBN (Electronic) | 9781538630815, 9781538630808 |
ISBN (Print) | 9781538630822 |
DOIs | |
Publication status | Published - 10 Sept 2018 |
Event | IEEE International Conference on Robotics and Automation 2018 - Brisbane Convention & Exhibition Centre, Brisbane, Australia Duration: 21 May 2018 → 25 May 2018 https://icra2018.org/ (Website) https://ieeexplore.ieee.org/xpl/conhome/8449910/proceeding (Proceedings) |
Publication series
Name | Proceedings - IEEE International Conference on Robotics and Automation |
---|---|
Publisher | IEEE, Institute of Electrical and Electronics Engineers |
ISSN (Print) | 1050-4729 |
Conference
Conference | IEEE International Conference on Robotics and Automation 2018 |
---|---|
Abbreviated title | ICRA 2018 |
Country/Territory | Australia |
City | Brisbane |
Period | 21/05/18 → 25/05/18 |
Internet address |
|