SceneCut: joint geometric and object segmentation for indoor scenes

Trung T. Pham, Thanh-Toan Do, Niko Sunderhauf, Ian Reid

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

32 Citations (Scopus)


This paper presents SceneCut, a novel approach to jointly discover previously unseen objects and non-object surfaces using a single RGB-D image. SceneCut's joint reasoning over scene semantics and geometry allows a robot to detect and segment object instances in complex scenes where modern deep learning-based methods either fail to separate object instances, or fail to detect objects that were not seen during training. SceneCut automatically decomposes a scene into meaningful regions which either represent objects or scene surfaces. The decomposition is qualified by an unified energy function over objectness and geometric fitting. We show how this energy function can be optimized efficiently by utilizing hierarchical segmentation trees. Moreover, we leverage a pre-trained convolutional oriented boundary network to predict accurate boundaries from images, which are used to construct high-quality region hierarchies. We evaluate SceneCut on several different indoor environments, and the results show that SceneCut significantly outperforms all the existing methods.

Original languageEnglish
Title of host publication2018 IEEE International Conference on Robotics and Automation (ICRA 2018)
EditorsPeter Corke, Nancy M Amato, Megan Emmons, Yoshihiko Nakamura, Markus Vincze
Place of PublicationPiscataway NJ USA
PublisherIEEE, Institute of Electrical and Electronics Engineers
Number of pages8
ISBN (Electronic)9781538630815, 9781538630808
ISBN (Print)9781538630822
Publication statusPublished - 2018
Externally publishedYes
EventIEEE International Conference on Robotics and Automation 2018 - Brisbane Convention & Exhibition Centre, Brisbane, Australia
Duration: 21 May 201825 May 2018 (Website) (Proceedings)

Publication series

NameProceedings - IEEE International Conference on Robotics and Automation
PublisherInstitute of Electrical and Electronics Engineers, Inc.
ISSN (Print)1050-4729
ISSN (Electronic)2577-087X


ConferenceIEEE International Conference on Robotics and Automation 2018
Abbreviated titleICRA 2018
Internet address

Cite this