Modeling caricature expressions by 3D blendshape and dynamic texture

Keyu Chen, Jianmin Zheng, Jianfei Cai, Juyong Zhang

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

2 Citations (Scopus)

Abstract

The problem of deforming an artist-drawn caricature according to a given normal face expression is of interest in applications such as social media, animation and entertainment. This paper presents a solution to the problem, with an emphasis on enhancing the ability to create desired expressions and meanwhile preserve the identity exaggeration style of the caricature, which imposes challenges due to the complicated nature of caricatures. The key of our solution is a novel method to model caricature expression, which extends traditional 3DMM representation to caricature domain. The method consists of shape modelling and texture generation for caricatures. Geometric optimization is developed to create identity-preserving blendshapes for reconstructing accurate and stable geometric shape, and a conditional generative adversarial network (cGAN) is designed for generating dynamic textures under target expressions. The combination of both shape and texture components makes the non-trivial expressions of a caricature be effectively defined by the extension of the popular 3DMM representation and a caricature can thus be flexibly deformed into arbitrary expressions with good results visually in both shape and color spaces. The experiments demonstrate the effectiveness of the proposed method.

Original languageEnglish
Title of host publicationProceedings of the 28th ACM International Conference on Multimedia
EditorsGuo-Jun Qi, Elisa Ricci, Zhengyou Zhang, Roger Zimmermann
Place of PublicationNew York NY USA
PublisherAssociation for Computing Machinery (ACM)
Pages3228-3236
Number of pages9
ISBN (Electronic)9781450379885
DOIs
Publication statusPublished - 2020
EventACM International Conference on Multimedia 2020 - Virtual, Online, United States of America
Duration: 12 Oct 202016 Oct 2020
Conference number: 28th
https://dl.acm.org/doi/proceedings/10.1145/3394171 (Proceedings)

Conference

ConferenceACM International Conference on Multimedia 2020
Abbreviated titleMM 2020
CountryUnited States of America
CityVirtual, Online
Period12/10/2016/10/20
Internet address

Keywords

  • blendshape
  • caricature
  • deformation
  • expression
  • texture

Cite this