Skip to main navigation Skip to search Skip to main content

DiffTV: Identity-Preserved Thermal-to-Visible Face Translation via Feature Alignment and Dual-Stage Conditions

  • Jingyu Lin
  • , Guiqin Zhao
  • , Jing Xu
  • , Guoli Wang
  • , Zejin Wang
  • , Antitza Dantcheva
  • , Lan Du
  • , Cunjian Chen

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

Abstract

The thermal-to-visible (T2V) face translation task is essential for enabling face verification in low-light or dark conditions by converting thermal infrared faces into their visible counterparts. However, this task faces two primary challenges. First, the inherent differences between the modalities hinder the effective use of thermal information to guide RGB face reconstruction. Second, translated RGB faces often lack the identity details of the corresponding visible faces, such as skin color. To tackle these challenges, we introduce DiffTV, the first Latent Diffusion Model (LDM) specifically designed for T2V facial image translation with a focus on preserving identity. Our approach proposes a novel heterogeneous feature alignment strategy that bridges the modal gap and extracts both coarse-and fine-grained identity features consistent with visible images. Furthermore, a dual-stage condition injection strategy introduces control information to guide identity-preserved translation. Experimental results demonstrate the superior performance of DiffTV, particularly in scenarios where maintaining identity integrity is critical.

Original languageEnglish
Title of host publicationProceedings of the 32nd ACM International Conference on Multimedia
EditorsYadan Luo, Toan Do, Yan Yan
Place of PublicationNew York NY USA
PublisherAssociation for Computing Machinery (ACM)
Pages10930-10938
Number of pages9
ISBN (Electronic)9798400706868
DOIs
Publication statusPublished - 2024
EventACM International Conference on Multimedia 2024 - Melbourne, Australia
Duration: 28 Oct 20241 Nov 2024
Conference number: 32nd
https://dl.acm.org/doi/book/10.1145/3664647 (Proceedings)
https://2024.acmmm.org/ (Website)

Conference

ConferenceACM International Conference on Multimedia 2024
Abbreviated titleMM 2024
Country/TerritoryAustralia
CityMelbourne
Period28/10/241/11/24
Internet address

Keywords

  • diffusion model
  • identity-preserving
  • image translation
  • thermal-to-visible

Cite this