T-FAKE: Synthesizing Thermal Images for Facial Landmarking

2024-08-27

arXiv.org

AI Summary

Existing facial datasets are heavily biased toward the RGB modality, while thermal imaging—despite its unique value in medical diagnostics, life sciences, and biometrics—lacks large-scale, keypoint-annotated thermal face data. To address this gap, we introduce T-FAKE, the first large-scale synthetic thermal facial dataset. We propose RGB2Thermal loss, which integrates Wasserstein distance with clinically grounded facial temperature statistics to achieve high-fidelity thermal style transfer. Additionally, we design a novel probabilistic keypoint prediction framework coupled with a label-adaptive network tailored for thermal imagery. Our method achieves significant improvements over baselines on both sparse (70-point) and dense (478-point) thermal facial keypoint detection tasks. All models are open-sourced and support plug-and-play deployment, facilitating reproducible research and practical adoption in thermal biometrics and clinical applications.

Technology Category

• Computer Vision: Biometrics, Face, Gesture & Pose
• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• Security and Privacy: Data transparency and provenance
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

Facial analysis is a key component in a wide range of applications such as security, autonomous driving, entertainment, and healthcare. Despite the availability of various facial RGB datasets, the thermal modality, which plays a crucial role in life sciences, medicine, and biometrics, has been largely overlooked. To address this gap, we introduce the T-FAKE dataset, a new large-scale synthetic thermal dataset with sparse and dense landmarks. To facilitate the creation of the dataset, we propose a novel RGB2Thermal loss function, which enables the transfer of thermal style to RGB faces. By utilizing the Wasserstein distance between thermal and RGB patches and the statistical analysis of clinical temperature distributions on faces, we ensure that the generated thermal images closely resemble real samples. Using RGB2Thermal style transfer based on our RGB2Thermal loss function, we create the T-FAKE dataset, a large-scale synthetic thermal dataset of faces. Leveraging our novel T-FAKE dataset, probabilistic landmark prediction, and label adaptation networks, we demonstrate significant improvements in landmark detection methods on thermal images across different landmark conventions. Our models show excellent performance with both sparse 70-point landmarks and dense 478-point landmark annotations. Our code and models are available at https://github.com/phflot/tfake.

Problem

• Lack of large-scale thermal facial datasets for analysis
• Need for accurate thermal image synthesis from RGB
• Improving landmark detection in thermal facial images

Innovation

• Introduces T-FAKE synthetic thermal dataset
• Proposes RGB2Thermal loss for style transfer
• Uses Wasserstein distance for realistic thermal generation