Robust real-time performance-driven 3D face tracking

Hai X. Pham, Vladimir Pavlovic, Jianfei Cai, Tat-jen Cham

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

5 Citations (Scopus)

Abstract

We introduce a novel robust hybrid 3D face tracking framework from RGBD video streams, which is capable of tracking head pose and facial actions without pre-calibration or intervention from a user. In particular, we emphasize on improving the tracking performance in instances where the tracked subject is at a large distance from the cameras, and the quality of point cloud deteriorates severely. This is accomplished by the combination of a flexible 3D shape regressor and the joint 2D+3D optimization on shape parameters. Our approach fits facial blendshapes to the point cloud of the human head, while being driven by an efficient and rapid 3D shape regressor trained on generic RGB datasets. As an on-line tracking system, the identity of the unknown user is adapted on-the-fly resulting in improved 3D model reconstruction and consequently better tracking performance. The result is a robust RGBD face tracker capable of handling a wide range of target scene depths, whose performances are demonstrated in our extensive experiments better than those of the state-of-the-arts.

Original languageEnglish
Title of host publication2016 23rd International Conference on Pattern Recognition (ICPR 2016)
EditorsLarry Davis, Alberto Del Bimbo, Brian C. Lovell
Place of PublicationPiscataway NJ USA
PublisherIEEE, Institute of Electrical and Electronics Engineers
Pages1851-1856
Number of pages6
ISBN (Electronic)9781509048472
ISBN (Print)9781509048489
DOIs
Publication statusPublished - 2016
Externally publishedYes
EventInternational Conference on Pattern Recognition 2016 - Cancun, Mexico
Duration: 4 Dec 20168 Dec 2016
Conference number: 23rd
http://www.icpr2016.org/site/

Conference

ConferenceInternational Conference on Pattern Recognition 2016
Abbreviated titleICPR 2016
CountryMexico
CityCancun
Period4/12/168/12/16
Internet address

Cite this

Pham, H. X., Pavlovic, V., Cai, J., & Cham, T. (2016). Robust real-time performance-driven 3D face tracking. In L. Davis, A. Del Bimbo, & B. C. Lovell (Eds.), 2016 23rd International Conference on Pattern Recognition (ICPR 2016) (pp. 1851-1856). [7899906] IEEE, Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/ICPR.2016.7899906