STAIR3D: simultaneous tracking and incremental registration for modeling 3D handheld objects

Krishneel Chaudhary, Xiangyu Chen, Wesley P. Chan, Kei Okada, Masayuki Inaba

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

1 Citation (Scopus)


Reconstructing a 3D model of an unknown object via incremental registration of multiple appearance models is a challenging task. With availability of low cost sensors and robust algorithms, the field of visual scene reconstruction has advanced considerably. While these advances has enabled robust reconstructions of cluttered and unstructured scenes, an active 3D reconstruction of a generic handheld object through registration of multiple appearance models from a single RGB-D viewpoint remains a difficult problem. The difficulty is maintaining the physical boundaries of an unknown object subjected to non-linear and unknown hand motion and appearance changes. As a consequence of these abrupt changes, the registration fails either due to motion drifting or boundary overflow. In this paper, we present a novel algorithm for active tracking and simultaneous registration of handheld objects subjected to abrupt changes in motion model. To reduce the effects of drifting in registration, a tracking algorithm with robust updating scheme is used to track the motion of the objects in real time. The updating algorithm is based on the idea of Gestalt principles formulated into Bayesian filter framework. The tracker predicted motion is used for object region segmentation and structured registration i.e., preregistration using the motion of tracker for initial alignment, fast feature based alignment and ICP based dense alignment. In this pipeline, we are able to compensate the effects of motion drifting and reject outliers from non-object regions while maintaining the boundary of the object.

Original languageEnglish
Title of host publication2017 IEEE International Conference on Advanced Intelligent Mechatronics (AIM 2017)
EditorsOliver Sawodny
Place of PublicationPiscataway NJ USA
PublisherIEEE, Institute of Electrical and Electronics Engineers
Number of pages8
ISBN (Electronic)9781509060009, 9781509059997
ISBN (Print)9781509060016, 9781509059980
Publication statusPublished - 21 Aug 2017
Externally publishedYes
EventIEEE/ASME International Conference on Advanced Intelligent Mechatronics 2017 - Munich, Germany
Duration: 3 Jul 20177 Jul 2017
Conference number: 16th (Conference website) (Proceedings)


ConferenceIEEE/ASME International Conference on Advanced Intelligent Mechatronics 2017
Abbreviated titleAIM 2017
Internet address


  • 3D-tracking
  • Convexity and symmetries
  • Incremental registration
  • Iterative closest point
  • Particle filters

Cite this