Deformation and correspondence aware unsupervised synthetic-to-real scene flow estimation for point clouds

Zhao Jin, Yinjie Lei, Naveed Akhtar, Haifeng Li, Munawar Hayat

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

21 Citations (Scopus)

Abstract

Point cloud scene flow estimation is of practical importance for dynamic scene navigation in autonomous driving. Since scene flow labels are hard to obtain, current methods train their models on synthetic data and transfer them to real scenes. However, large disparities between existing synthetic datasets and real scenes lead to poor model transfer. We make two major contributions to address that. First, we develop a point cloud collector and scene flow annotator for GTA-V engine to automatically obtain diverse realistic training samples without human intervention. With that, we develop a large-scale synthetic scene flow dataset GTA-SF. Second, we propose a mean-teacher-based domain adaptation framework that leverages self-generated pseudo-labels of the target domain. It also explicitly incorporates shape deformation regularization and surface correspondence refinement to address distortions and misalignments in domain transfer. Through extensive experiments, we show that our GTA-SF dataset leads to a consistent boost in model generalization to three real datasets (i.e., Waymo, Lyft and KITTI) as compared to the most widely used FT3D dataset. Moreover, our framework achieves superior adaptation performance on six source-target dataset pairs, remarkably closing the average domain gap by 60%. Data and codes are available at https://github.com/leolyj/DCA-SRSFE

Original languageEnglish
Title of host publicationProceedings - 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022
EditorsKosta Derpanis
Place of PublicationPiscataway NJ USA
PublisherIEEE, Institute of Electrical and Electronics Engineers
Pages7223-7233
Number of pages11
ISBN (Electronic)9781665469463
ISBN (Print)9781665469470
DOIs
Publication statusPublished - 2022
EventIEEE Conference on Computer Vision and Pattern Recognition 2022 - New Orleans, United States of America
Duration: 19 Jun 202224 Jun 2022
https://ieeexplore.ieee.org/xpl/conhome/9878378/proceeding (Proceedings)
https://cvpr2022.thecvf.com
https://cvpr2022.thecvf.com/ (Website)

Conference

ConferenceIEEE Conference on Computer Vision and Pattern Recognition 2022
Abbreviated titleCVPR 2022
Country/TerritoryUnited States of America
CityNew Orleans
Period19/06/2224/06/22
Internet address

Keywords

  • Deep learning architectures and techniques
  • Motion and tracking
  • Robot vision
  • Scene analysis and understanding
  • Transfer/low-shot/long-tail learning

Cite this