Depth estimation from single image and semantic prior

Praful Hambarde, Akshay Dudhane, Prashant W. Patil, Subrahmanyam Murala, Abhinav Dhall

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

33 Citations (Scopus)


The multi-modality sensor fusion technique is an active research area in scene understating. In this work, we explore the RGB image and semantic-map fusion methods for depth estimation. The LiDARs, Kinect, and TOF depth sensors are unable to predict the depth-map at illuminate and monotonous pattern surface. In this paper, we propose a semantic-to-depth generative adversarial network (S2D-GAN) for depth estimation from RGB image and its semantic-map. In the first stage, the proposed S2D-GAN estimates the coarse level depthmap using a semantic-to-coarse-depth generative adversarial network (S2CD-GAN) while the second stage estimates the fine-level depth-map using a cascaded multi-scale spatial pooling network. The experimental analysis of the proposed S2D-GAN performed on NYU-Depth-V2 dataset shows that the proposed S2D-GAN gives outstanding result over existing single image depth estimation and RGB with sparse samples methods. The proposed S2D-GAN also gives efficient results on the real-world indoor and outdoor image depth estimation.

Original languageEnglish
Title of host publication2020 IEEE International Conference on Image Processing, ICIP 2020 - Proceedings
Place of PublicationPiscataway NJ USA
PublisherIEEE, Institute of Electrical and Electronics Engineers
Number of pages5
ISBN (Electronic)9781728163956, 9781728163949
ISBN (Print)9781728163963
Publication statusPublished - 2020
EventIEEE International Conference on Image Processing 2020 - Virtual, Abu Dhabi, United Arab Emirates
Duration: 25 Sept 202028 Sept 2020
Conference number: 27th (Proceedings) (Website)

Publication series

NameProceedings - International Conference on Image Processing, ICIP
PublisherThe Institute of Electrical and Electronics Engineers, Inc.
ISSN (Print)1522-4880
ISSN (Electronic)2381-8549


ConferenceIEEE International Conference on Image Processing 2020
Abbreviated titleICIP 2020
Country/TerritoryUnited Arab Emirates
CityAbu Dhabi
Internet address


  • Coarse-level depth-map
  • Depth estimation
  • Fine-level depth-map
  • Semantic map
  • Single image

Cite this