Weakly-supervised 3D hand pose estimation from monocular RGB images

Yujun Cai, Liuhao Ge, Jianfei Cai, Junsong Yuan

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

21 Citations (Scopus)

Abstract

Compared with depth-based 3D hand pose estimation, it is more challenging to infer 3D hand pose from monocular RGB images, due to substantial depth ambiguity and the difficulty of obtaining fully-annotated training data. Different from existing learning-based monocular RGB-input approaches that require accurate 3D annotations for training, we propose to leverage the depth images that can be easily obtained from commodity RGB-D cameras during training, while during testing we take only RGB inputs for 3D joint predictions. In this way, we alleviate the burden of the costly 3D annotations in real-world dataset. Particularly, we propose a weakly-supervised method, adaptating from fully-annotated synthetic dataset to weakly-labeled real-world dataset with the aid of a depth regularizer, which generates depth maps from predicted 3D pose and serves as weak supervision for 3D pose regression. Extensive experiments on benchmark datasets validate the effectiveness of the proposed depth regularizer in both weakly-supervised and fully-supervised settings.

Original languageEnglish
Title of host publicationComputer Vision – ECCV 2018
Subtitle of host publication15th European Conference Munich, Germany, September 8–14, 2018 Proceedings, Part VI
EditorsVittorio Ferrari, Martial Hebert, Cristian Sminchisescu, Yair Weiss
Place of PublicationCham Switzerland
PublisherSpringer
Pages678-694
Number of pages17
ISBN (Electronic)9783030012311
ISBN (Print)9783030012304
DOIs
Publication statusPublished - 2018
Externally publishedYes
EventEuropean Conference on Computer Vision 2018 - Munich, Germany
Duration: 8 Sep 201814 Sep 2018
Conference number: 15th
https://eccv2018.org/
https://link.springer.com/book/10.1007/978-3-030-01246-5 (Proceedings)

Publication series

NameLecture Notes in Computer Science
PublisherSpringer
Volume11210
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

ConferenceEuropean Conference on Computer Vision 2018
Abbreviated titleECCV 2018
Country/TerritoryGermany
CityMunich
Period8/09/1814/09/18
Internet address

Keywords

  • 3D hand pose estimation
  • Depth regularizer
  • Weakly-supervised methods

Cite this