STAIR3D: simultaneous tracking and incremental registration for modeling 3D handheld objects

Reconstructing a 3D model of an unknown object via incremental registration of multiple appearance models is a challenging task. With availability of low cost sensors and robust algorithms, the field of visual scene reconstruction has advanced considerably. While these advances has enabled robust reconstructions of cluttered and unstructured scenes, an active 3D reconstruction of a generic handheld object through registration of multiple appearance models from a single RGB-D viewpoint remains a difficult problem. The difficulty is maintaining the physical boundaries of an unknown object subjected to non-linear and unknown hand motion and appearance changes. As a consequence of these abrupt changes, the registration fails either due to motion drifting or boundary overflow. In this paper, we present a novel algorithm for active tracking and simultaneous registration of handheld objects subjected to abrupt changes in motion model. To reduce the effects of drifting in registration, a tracking algorithm with robust updating scheme is used to track the motion of the objects in real time. The updating algorithm is based on the idea of Gestalt principles formulated into Bayesian filter framework. The tracker predicted motion is used for object region segmentation and structured registration i.e., preregistration using the motion of tracker for initial alignment, fast feature based alignment and ICP based dense alignment. In this pipeline, we are able to compensate the effects of motion drifting and reject outliers from non-object regions while maintaining the boundary of the object.