3D contact point cloud reconstruction from vision-based tactile flow

Yipai Du, Guanlan Zhang, Michael Yu Wang

Research output: Contribution to journalArticleResearchpeer-review

Abstract

With the growing interest in vision-based tactile sensors, various types of sensors that utilize digital imaging are being developed. Among them, a group of sensors captures the tactile flow resulted from the contact deformation using the optical flow algorithm from computer vision and achieves full resolution deformation tracking on the tactile surface. In this work, a novel 3D contact reconstruction algorithm is proposed and evaluated. It exploits the contact geometry and projection relationship in the tactile flow, which are versatile for vision-based tactile sensors, unique for tactile perception but not inherited from computer vision. The resulted 3D contact point cloud representation is consistent with the tactile flow constraint, scale estimation, and contact edge estimation. It can be directly manipulated in downstream applications such as contact force estimation and contact pose estimation. Experiments and examples are provided that indicate the potential for the proposed tactile processing algorithm to connect tactile perception to tactile enabled robotic manipulation tasks.

Original languageEnglish
Pages (from-to)12177-12184
Number of pages8
JournalIEEE Robotics and Automation Letters
Volume7
Issue number4
DOIs
Publication statusPublished - Oct 2022
Externally publishedYes

Keywords

  • Contact modeling
  • force and tactile sensing
  • perception for grasping and manipulation

Cite this