AffordanceNet: an end-to-end deep learning approach for object affordance detection

Thanh-Toan Do, Anh Nguyen, Ian Reid

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

41 Citations (Scopus)

Abstract

We propose AffordanceNet, a new deep learning approach to simultaneously detect multiple objects and their affordances from RGB images. Our AffordanceNet has two branches: an object detection branch to localize and classify the object, and an affordance detection branch to assign each pixel in the object to its most probable affordance label. The proposed framework employs three key components for effectively handling the multiclass problem in the affordance mask: a sequence of deconvolutional layers, a robust resizing strategy, and a multi-task loss function. The experimental results on the public datasets show that our AffordanceNet outperforms recent state-of-the-art methods by a fair margin, while its end-to-end architecture allows the inference at the speed of 150ms per image. This makes our AffordanceNet well suitable for real-time robotic applications. Furthermore, we demonstrate the effectiveness of AffordanceNet in different testing environments and in real robotic applications. The source code is available at https://github.com/nqanh/affordance-net.

Original languageEnglish
Title of host publication2018 IEEE International Conference on Robotics and Automation (ICRA 2018)
EditorsPeter Corke, Nancy M Amato, Megan Emmons, Yoshihiko Nakamura, Markus Vincze
Place of PublicationPiscataway NJ USA
PublisherIEEE, Institute of Electrical and Electronics Engineers
Pages5882-5889
Number of pages8
ISBN (Electronic)9781538630815, 9781538630808
ISBN (Print)9781538630822
DOIs
Publication statusPublished - 2018
Externally publishedYes
EventIEEE International Conference on Robotics and Automation 2018 - Brisbane Convention & Exhibition Centre, Brisbane, Australia
Duration: 21 May 201825 May 2018
https://icra2018.org/ (Website)
https://ieeexplore.ieee.org/xpl/conhome/8449910/proceeding (Proceedings)

Publication series

NameProceedings - IEEE International Conference on Robotics and Automation
PublisherInstitute of Electrical and Electronics Engineers, Inc.
ISSN (Print)1050-4729
ISSN (Electronic)2577-087X

Conference

ConferenceIEEE International Conference on Robotics and Automation 2018
Abbreviated titleICRA 2018
CountryAustralia
CityBrisbane
Period21/05/1825/05/18
Internet address

Cite this