FaceCollage: a rapidly deployable system for real-time head reconstruction for on-the-go 3D telepresence

Fuwen Tan, Chi Wing Fu, Teng Deng, Jianfei Cai, Tat Jen Cham

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

4 Citations (Scopus)

Abstract

This paper presents FaceCollage, a robust and real-time system for head reconstruction that can be used to create easy-to-deploy telepresence systems, using a pair of consumer-grade RGBD cameras that provide a wide range of views of the reconstructed user. A key feature is that the system is very simple to rapidly deploy, with autonomous calibration and requiring minimal intervention from the user, other than casually placing the cameras. This system is realized through three technical contributions: (1) a fully automatic calibration method, which analyzes and correlates the left and right RGBD faces just by the face features; (2) an implementation that exploits the parallel computation capability of GPU throughout most of the system pipeline, in order to attain real-time performance; and (3) a complete integrated system on which we conducted various experiments to demonstrate its capability, robustness, and performance, including testing the system on twelve participants with visually-pleasing results.

Original languageEnglish
Title of host publicationMM’17 - Proceedings of the 2017 ACM Multimedia Conference
EditorsKuan-Ta Chen, Susanne Boll, Phoebe Chen, Gerald Friedland, Jia Li, Shuicheng Yan
Place of PublicationNew York NY USA
PublisherAssociation for Computing Machinery (ACM)
Pages64-72
Number of pages9
ISBN (Electronic)9781450349062
DOIs
Publication statusPublished - 2017
EventACM International Conference on Multimedia 2017 - Mountain View, United States of America
Duration: 23 Oct 201727 Oct 2017
Conference number: 25th
https://dl.acm.org/doi/proceedings/10.1145/3123266

Conference

ConferenceACM International Conference on Multimedia 2017
Abbreviated titleMM 2017
CountryUnited States of America
CityMountain View
Period23/10/1727/10/17
Internet address

Keywords

  • 3D telepresence
  • Face capture
  • RGBD sensors

Cite this