Recovering faces from portraits with auxiliary facial attributes

Fatemeh Shiri, Xin Yu, Fatih Porikli, Richard Hartley, Piotr Koniusz

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

7 Citations (Scopus)

Abstract

Recovering a photorealistic face from an artistic portrait is a challenging task since crucial facial details are often distorted or completely lost in artistic compositions. To handle this loss, we propose an Attribute-guided Face Recovery from Portraits (AFRP) that utilizes a Face Recovery Network (FRN) and a Discriminative Network (DN). FRN consists of an autoencoder with residual block-embedded skip-connections and incorporates facial attribute vectors into the feature maps of input portraits at the bottleneck of the autoencoder. DN has multiple convolutional and fully-connected layers, and its role is to enforce FRN to generate authentic face images with corresponding facial attributes dictated by the input attribute vectors. For the preservation of identities, we impose the recovered and ground-truth faces to share similar visual features. Specifically, DN determines whether the recovered image looks like a real face and checks if the facial attributes extracted from the recovered image are consistent with given attributes. Our method can recover photorealistic identity-preserving faces with desired attributes from unseen stylized portraits, artistic paintings, and hand-drawn sketches. On large-scale synthesized and sketch datasets, we demonstrate that our face recovery method achieves state-of-the-art results.

Original languageEnglish
Title of host publicationProceedings - 2019 IEEE Winter Conference on Applications of Computer Vision, WACV 2018
EditorsMichael Brown, Yanxi Liu, Peyman Milanfar
Place of PublicationPiscataway NJ USA
PublisherIEEE, Institute of Electrical and Electronics Engineers
Pages406-415
Number of pages10
ISBN (Electronic)9781728119755
ISBN (Print)9781728119762
DOIs
Publication statusPublished - 2019
Externally publishedYes
EventIEEE Winter Conference on Applications of Computer Vision 2018 - Lake Tahoe, United States of America
Duration: 12 Mar 201815 Mar 2018
https://wacv18.wacv.net
https://ieeexplore.ieee.org/xpl/conhome/8345804/proceeding (Proceedings)

Conference

ConferenceIEEE Winter Conference on Applications of Computer Vision 2018
Abbreviated titleWACV 2018
Country/TerritoryUnited States of America
CityLake Tahoe
Period12/03/1815/03/18
Internet address

Cite this