Abstract
Understanding the 3D structure of a scene is of vital importance, when it comes to developing fully autonomous robots. To this end, we present a novel deep learning based framework that estimates depth, surface normals and surface curvature by only using a single RGB image. To the best of our knowledge this is the first work to estimate surface curvature from colour using a machine learning approach. Additionally, we demonstrate that by tuning the network to infer well designed features, such as surface curvature, we can achieve improved performance at estimating depth and normals. This indicates that network guidance is still a useful aspect of designing and training a neural network. We run extensive experiments where the network is trained to infer different tasks while the model capacity is kept constant resulting in different feature maps based on the tasks at hand. We outperform the previous state-of-the-art benchmarks which jointly estimate depths and surface normals while predicting surface curvature in parallel.
| Original language | English |
|---|---|
| Title of host publication | 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2017) |
| Editors | Richard Vaughan |
| Place of Publication | Piscataway NJ USA |
| Publisher | IEEE, Institute of Electrical and Electronics Engineers |
| Pages | 1505-1512 |
| Number of pages | 8 |
| ISBN (Electronic) | 9781538626825, 9781538626818 |
| ISBN (Print) | 9781538626832 |
| DOIs | |
| Publication status | Published - 13 Dec 2017 |
| Event | IEEE/RSJ International Conference on Intelligent Robots and Systems 2017: Friendly People, Friendly Robots - Vancouver, Canada Duration: 24 Sep 2017 → 28 Sep 2017 Conference number: 30th http://iros2017.org/ |
Publication series
| Name | IEEE International Conference on Intelligent Robots and Systems |
|---|---|
| Publisher | IEEE, Institute of Electrical and Electronics Engineers |
| ISSN (Print) | 2153-0858 |
| ISSN (Electronic) | 2153-0866 |
Conference
| Conference | IEEE/RSJ International Conference on Intelligent Robots and Systems 2017 |
|---|---|
| Abbreviated title | IROS2017 |
| Country | Canada |
| City | Vancouver |
| Period | 24/09/17 → 28/09/17 |
| Internet address |
Cite this
}
Joint prediction of depths, normals and surface curvature from RGB images using CNNs. / Dharmasiri, Thanuja; Spek, Andrew; Drummond, Tom.
2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2017). ed. / Richard Vaughan. Piscataway NJ USA : IEEE, Institute of Electrical and Electronics Engineers, 2017. p. 1505-1512 (IEEE International Conference on Intelligent Robots and Systems).Research output: Chapter in Book/Report/Conference proceeding › Conference Paper › Research › peer-review
TY - GEN
T1 - Joint prediction of depths, normals and surface curvature from RGB images using CNNs
AU - Dharmasiri, Thanuja
AU - Spek, Andrew
AU - Drummond, Tom
PY - 2017/12/13
Y1 - 2017/12/13
N2 - Understanding the 3D structure of a scene is of vital importance, when it comes to developing fully autonomous robots. To this end, we present a novel deep learning based framework that estimates depth, surface normals and surface curvature by only using a single RGB image. To the best of our knowledge this is the first work to estimate surface curvature from colour using a machine learning approach. Additionally, we demonstrate that by tuning the network to infer well designed features, such as surface curvature, we can achieve improved performance at estimating depth and normals. This indicates that network guidance is still a useful aspect of designing and training a neural network. We run extensive experiments where the network is trained to infer different tasks while the model capacity is kept constant resulting in different feature maps based on the tasks at hand. We outperform the previous state-of-the-art benchmarks which jointly estimate depths and surface normals while predicting surface curvature in parallel.
AB - Understanding the 3D structure of a scene is of vital importance, when it comes to developing fully autonomous robots. To this end, we present a novel deep learning based framework that estimates depth, surface normals and surface curvature by only using a single RGB image. To the best of our knowledge this is the first work to estimate surface curvature from colour using a machine learning approach. Additionally, we demonstrate that by tuning the network to infer well designed features, such as surface curvature, we can achieve improved performance at estimating depth and normals. This indicates that network guidance is still a useful aspect of designing and training a neural network. We run extensive experiments where the network is trained to infer different tasks while the model capacity is kept constant resulting in different feature maps based on the tasks at hand. We outperform the previous state-of-the-art benchmarks which jointly estimate depths and surface normals while predicting surface curvature in parallel.
UR - http://www.scopus.com/inward/record.url?scp=85041946693&partnerID=8YFLogxK
U2 - 10.1109/IROS.2017.8205954
DO - 10.1109/IROS.2017.8205954
M3 - Conference Paper
AN - SCOPUS:85041946693
SN - 9781538626832
T3 - IEEE International Conference on Intelligent Robots and Systems
SP - 1505
EP - 1512
BT - 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2017)
A2 - Vaughan, Richard
PB - IEEE, Institute of Electrical and Electronics Engineers
CY - Piscataway NJ USA
ER -