Joint registration and representation learning for unconstrained face identification

Munawar Hayat, Salman H. Khan, Naoufel Werghi, Roland Goecke

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

15 Citations (Scopus)


Recent advances in deep learning have resulted in human-level performances on popular unconstrained face datasets including Labeled Faces in the Wild and YouTube Faces. To further advance research, IJB-A benchmark was recently introduced with more challenges especially in the form of extreme head poses. Registration of such faces is quite demanding and often requires laborious procedures like facial landmark localization. In this paper, we propose a Convolutional Neural Networks based data-driven approach which learns to simultaneously register and represent faces. We validate the proposed scheme on template based unconstrained face identification. Here, a template contains multiple media in the form of images and video frames. Unlike existing methods which synthesize all template media information at feature level, we propose to keep the template media intact. Instead, we represent gallery templates by their trained one-vs-rest discriminative models and then employ a Bayesian strategy which optimally fuses decisions of all medias in a query template. We demonstrate the efficacy of the proposed scheme on IJB-A, YouTube Celebrities and COX datasets where our approach achieves significant relative performance boosts of 3.6%, 21.6% and 12.8% respectively.

Original languageEnglish
Title of host publicationProceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017
EditorsYanxi Liu, James M. Rehg, Camillo J. Taylor, Ying Wu
Place of PublicationPiscataway NJ USA
PublisherIEEE, Institute of Electrical and Electronics Engineers
Number of pages10
ISBN (Electronic)9781538604571
ISBN (Print)9781538604588
Publication statusPublished - 2017
Externally publishedYes
EventIEEE Conference on Computer Vision and Pattern Recognition 2017 - Honolulu, United States of America
Duration: 21 Jul 201726 Jul 2017 (Proceedings)


ConferenceIEEE Conference on Computer Vision and Pattern Recognition 2017
Abbreviated titleCVPR 2017
CountryUnited States of America
Internet address

Cite this