CReaM: Condensed Real-time Models for depth prediction using Convolutional Neural Networks

Andrew Spek, Thanuja Dharmasiri, Tom Drummond

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

2 Citations (Scopus)

Abstract

Since the resurgence of CNNs the robotic vision community has developed a range of algorithms that perform classification, semantic segmentation and structure prediction (depths, normals, surface curvature) using neural networks. While some of these models achieve state-of-the art results and super human level performance, deploying these models in a time critical robotic environment remains an ongoing challenge. Real-time frameworks are of paramount importance to build a robotic society where humans and robots integrate seamlessly. To this end, we present a novel real-time structure prediction framework that predicts depth at 30 frames per second on an NVIDIA-TX2. At the time of writing, this is the first piece of work to showcase such a capability on a mobile platform. We also demonstrate with extensive experiments that neural networks with very large model capacities can be leveraged in order to train accurate condensed model architectures in a 'from teacher to student' style knowledge transfer.

Original languageEnglish
Title of host publication2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
EditorsCecilia Laschi
Place of PublicationPiscataway NJ USA
PublisherIEEE, Institute of Electrical and Electronics Engineers
Pages540-547
Number of pages8
ISBN (Electronic)9781538680940, 9781538680933
ISBN (Print)9781538680957
DOIs
Publication statusPublished - 27 Dec 2018
EventIEEE/RSJ International Conference on Intelligent Robots and Systems 2018 - Madrid, Spain
Duration: 1 Oct 20185 Oct 2018
https://www.iros2018.org/

Publication series

NameIEEE International Conference on Intelligent Robots and Systems
PublisherIEEE, Institute of Electrical and Electronics Engineers
ISSN (Print)2153-0858
ISSN (Electronic)2153-0866

Conference

ConferenceIEEE/RSJ International Conference on Intelligent Robots and Systems 2018
Abbreviated titleIROS 2018
CountrySpain
CityMadrid
Period1/10/185/10/18
Internet address

Cite this

Spek, A., Dharmasiri, T., & Drummond, T. (2018). CReaM: Condensed Real-time Models for depth prediction using Convolutional Neural Networks. In C. Laschi (Ed.), 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (pp. 540-547). (IEEE International Conference on Intelligent Robots and Systems). Piscataway NJ USA: IEEE, Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/IROS.2018.8594243
Spek, Andrew ; Dharmasiri, Thanuja ; Drummond, Tom. / CReaM : Condensed Real-time Models for depth prediction using Convolutional Neural Networks. 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). editor / Cecilia Laschi. Piscataway NJ USA : IEEE, Institute of Electrical and Electronics Engineers, 2018. pp. 540-547 (IEEE International Conference on Intelligent Robots and Systems).
@inproceedings{65fd9189aa2a4c4a820d8162c52b9438,
title = "CReaM: Condensed Real-time Models for depth prediction using Convolutional Neural Networks",
abstract = "Since the resurgence of CNNs the robotic vision community has developed a range of algorithms that perform classification, semantic segmentation and structure prediction (depths, normals, surface curvature) using neural networks. While some of these models achieve state-of-the art results and super human level performance, deploying these models in a time critical robotic environment remains an ongoing challenge. Real-time frameworks are of paramount importance to build a robotic society where humans and robots integrate seamlessly. To this end, we present a novel real-time structure prediction framework that predicts depth at 30 frames per second on an NVIDIA-TX2. At the time of writing, this is the first piece of work to showcase such a capability on a mobile platform. We also demonstrate with extensive experiments that neural networks with very large model capacities can be leveraged in order to train accurate condensed model architectures in a 'from teacher to student' style knowledge transfer.",
author = "Andrew Spek and Thanuja Dharmasiri and Tom Drummond",
year = "2018",
month = "12",
day = "27",
doi = "10.1109/IROS.2018.8594243",
language = "English",
isbn = "9781538680957",
series = "IEEE International Conference on Intelligent Robots and Systems",
publisher = "IEEE, Institute of Electrical and Electronics Engineers",
pages = "540--547",
editor = "Cecilia Laschi",
booktitle = "2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)",
address = "United States of America",

}

Spek, A, Dharmasiri, T & Drummond, T 2018, CReaM: Condensed Real-time Models for depth prediction using Convolutional Neural Networks. in C Laschi (ed.), 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE International Conference on Intelligent Robots and Systems, IEEE, Institute of Electrical and Electronics Engineers, Piscataway NJ USA, pp. 540-547, IEEE/RSJ International Conference on Intelligent Robots and Systems 2018, Madrid, Spain, 1/10/18. https://doi.org/10.1109/IROS.2018.8594243

CReaM : Condensed Real-time Models for depth prediction using Convolutional Neural Networks. / Spek, Andrew; Dharmasiri, Thanuja; Drummond, Tom.

2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). ed. / Cecilia Laschi. Piscataway NJ USA : IEEE, Institute of Electrical and Electronics Engineers, 2018. p. 540-547 (IEEE International Conference on Intelligent Robots and Systems).

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

TY - GEN

T1 - CReaM

T2 - Condensed Real-time Models for depth prediction using Convolutional Neural Networks

AU - Spek, Andrew

AU - Dharmasiri, Thanuja

AU - Drummond, Tom

PY - 2018/12/27

Y1 - 2018/12/27

N2 - Since the resurgence of CNNs the robotic vision community has developed a range of algorithms that perform classification, semantic segmentation and structure prediction (depths, normals, surface curvature) using neural networks. While some of these models achieve state-of-the art results and super human level performance, deploying these models in a time critical robotic environment remains an ongoing challenge. Real-time frameworks are of paramount importance to build a robotic society where humans and robots integrate seamlessly. To this end, we present a novel real-time structure prediction framework that predicts depth at 30 frames per second on an NVIDIA-TX2. At the time of writing, this is the first piece of work to showcase such a capability on a mobile platform. We also demonstrate with extensive experiments that neural networks with very large model capacities can be leveraged in order to train accurate condensed model architectures in a 'from teacher to student' style knowledge transfer.

AB - Since the resurgence of CNNs the robotic vision community has developed a range of algorithms that perform classification, semantic segmentation and structure prediction (depths, normals, surface curvature) using neural networks. While some of these models achieve state-of-the art results and super human level performance, deploying these models in a time critical robotic environment remains an ongoing challenge. Real-time frameworks are of paramount importance to build a robotic society where humans and robots integrate seamlessly. To this end, we present a novel real-time structure prediction framework that predicts depth at 30 frames per second on an NVIDIA-TX2. At the time of writing, this is the first piece of work to showcase such a capability on a mobile platform. We also demonstrate with extensive experiments that neural networks with very large model capacities can be leveraged in order to train accurate condensed model architectures in a 'from teacher to student' style knowledge transfer.

UR - http://www.scopus.com/inward/record.url?scp=85063011950&partnerID=8YFLogxK

U2 - 10.1109/IROS.2018.8594243

DO - 10.1109/IROS.2018.8594243

M3 - Conference Paper

AN - SCOPUS:85063011950

SN - 9781538680957

T3 - IEEE International Conference on Intelligent Robots and Systems

SP - 540

EP - 547

BT - 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

A2 - Laschi, Cecilia

PB - IEEE, Institute of Electrical and Electronics Engineers

CY - Piscataway NJ USA

ER -

Spek A, Dharmasiri T, Drummond T. CReaM: Condensed Real-time Models for depth prediction using Convolutional Neural Networks. In Laschi C, editor, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Piscataway NJ USA: IEEE, Institute of Electrical and Electronics Engineers. 2018. p. 540-547. (IEEE International Conference on Intelligent Robots and Systems). https://doi.org/10.1109/IROS.2018.8594243