Fused geometry augmented images for analyzing textured mesh

Bilal Taha, Munawar Hayat, Stefano Berretti, Naoufel Werghi

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearch

Abstract

In this paper, we propose a novel multi-modal mesh surface representation fusing texture and geometric data. Our approach defines an inverse mapping between different geometric descriptors computed on the mesh surface or its down-sampled version, and the corresponding 2D texture image of the mesh, allowing the construction of fused geometrically augmented images. This new fused modality enables us to learn feature representations from 3D data in a highly efficient manner by simply employing standard convolutional neural networks in a transfer-learning mode. In contrast to existing methods, the proposed approach is both computationally and memory efficient, preserves intrinsic geometric information and learns highly discriminative feature representation by effectively fusing shape and texture information at data level. The efficacy of our approach is demonstrated for the tasks of facial action unit detection, expression classification, and skin lesion classification, showing competitive performance with state of the art methods.

Original languageEnglish
Title of host publicationSmart Multimedia
Subtitle of host publicationSecond International Conference, ICSM 2019 San Diego, CA, USA, December 16–18, 2019 Revised Selected Papers
EditorsTroy McDaniel, Stefano Berretti, Igor D.D. Curcio, Anup Basu
Place of PublicationCham Switzerland
PublisherSpringer
Pages3-12
Number of pages10
ISBN (Electronic)9783030544072
ISBN (Print)9783030544065
DOIs
Publication statusPublished - 2020
Externally publishedYes
EventInternational Conference on Smart Multimedia 2019 - San Diego, United States of America
Duration: 16 Dec 201918 Dec 2019
Conference number: 2nd
https://link.springer.com/book/10.1007/978-3-030-54407-2 (Website)
https://smartmultimedia.org/2019/ (Website)

Publication series

NameLecture Notes in Computer Science
PublisherSpringer
Volume12015
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

ConferenceInternational Conference on Smart Multimedia 2019
Abbreviated titleICSM 2019
CountryUnited States of America
CitySan Diego
Period16/12/1918/12/19
Internet address

Keywords

  • Image representation
  • Learned features
  • Mesh surface analysis
  • Surface classification

Cite this