Using 3D face priors for depth recovery

Chongyu Chen, Hai Xuan Pham, Vladimir Pavlovic, Jianfei Cai, Guangming Shi, Yuefang Gao, Hui Cheng

Research output: Contribution to journalArticleResearchpeer-review

1 Citation (Scopus)

Abstract

For repairing inaccurate depth measurements from commodity RGB-D sensors, existing depth recovery methods primarily rely on low-level and rigid prior information. However, as the depth quality deteriorates, the recovered depth maps become increasingly unreliable, especially for non-rigid objects. Thus, additional high-level and non-rigid information is needed to improve the recovery quality. Taking as a starting point the human face that is the primary prior available in many high-level tasks, in this paper, we incorporate face priors into the depth recovery process. In particular, we propose a joint optimization framework that consists of two main steps: transforming the face model for better alignment and applying face priors for improved depth recovery. Face priors from both sparse and dense 3D face models are studied. By comparing with the baseline method on benchmark datasets, we demonstrate that the proposed method can achieve up to 23.8% improvement in depth recovery with more accurate face registrations, bringing inspirations to both non-rigid object modeling and analysis.

Original languageEnglish
Pages (from-to)16-29
Number of pages14
JournalJournal of Visual Communication and Image Representation
Volume48
DOIs
Publication statusPublished - Oct 2017
Externally publishedYes

Keywords

  • Depth recovery
  • Face model
  • Image restoration

Cite this