Joint prediction of depths, normals and surface curvature from RGB images using CNNs

Thanuja Dharmasiri, Andrew Spek, Tom Drummond

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

11 Citations (Scopus)

Abstract

Understanding the 3D structure of a scene is of vital importance, when it comes to developing fully autonomous robots. To this end, we present a novel deep learning based framework that estimates depth, surface normals and surface curvature by only using a single RGB image. To the best of our knowledge this is the first work to estimate surface curvature from colour using a machine learning approach. Additionally, we demonstrate that by tuning the network to infer well designed features, such as surface curvature, we can achieve improved performance at estimating depth and normals. This indicates that network guidance is still a useful aspect of designing and training a neural network. We run extensive experiments where the network is trained to infer different tasks while the model capacity is kept constant resulting in different feature maps based on the tasks at hand. We outperform the previous state-of-the-art benchmarks which jointly estimate depths and surface normals while predicting surface curvature in parallel.

Original languageEnglish
Title of host publication2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2017)
EditorsRichard Vaughan
Place of PublicationPiscataway NJ USA
PublisherIEEE, Institute of Electrical and Electronics Engineers
Pages1505-1512
Number of pages8
ISBN (Electronic)9781538626825, 9781538626818
ISBN (Print)9781538626832
DOIs
Publication statusPublished - 13 Dec 2017
EventIEEE/RSJ International Conference on Intelligent Robots and Systems 2017: Friendly People, Friendly Robots - Vancouver, Canada
Duration: 24 Sep 201728 Sep 2017
Conference number: 30th
http://iros2017.org/

Publication series

NameIEEE International Conference on Intelligent Robots and Systems
PublisherIEEE, Institute of Electrical and Electronics Engineers
ISSN (Print)2153-0858
ISSN (Electronic)2153-0866

Conference

ConferenceIEEE/RSJ International Conference on Intelligent Robots and Systems 2017
Abbreviated titleIROS2017
CountryCanada
CityVancouver
Period24/09/1728/09/17
Internet address

Cite this