DFREC: DeepFake Identity Recovery Based on Identity-aware Masked Autoencoder

2024-12-10

arXiv.org

AI Summary

Existing deepfake detection methods exhibit strong generalization but lack interpretability and identity traceability, thereby limiting forensic investigation efficacy. To address this, we propose the first high-fidelity dual-face restoration framework tailored to single forged images, enabling simultaneous reconstruction of both source and target faces to support identity attribution and attack forensics. Methodologically, we design an identity-aware masked autoencoder architecture, introducing a novel Identity Segmentation Module (ISM) to disentangle forged content, and developing Source and Target Identity Reconstruction Modules (SIRM/TIRM) for collaborative dual-path reconstruction. Our approach achieves state-of-the-art performance on FaceForensics++, CelebAMegaFS, and FFHQ-E4S, significantly outperforming prior restoration methods. It demonstrates robust reconstruction across six high-fidelity face-swapping attacks, with consistent improvements in PSNR (+2.1–3.7 dB) and SSIM (+0.028–0.051).

Technology Category

• Computer Vision: Adversarial Attacks & Robustness
• Machine Learning: Adversarial Learning & Robustness
• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

Recent advances in deepfake forensics have primarily focused on improving the classification accuracy and generalization performance. Despite enormous progress in detection accuracy across a wide variety of forgery algorithms, existing algorithms lack intuitive interpretability and identity traceability to help with forensic investigation. In this paper, we introduce a novel DeepFake Identity Recovery scheme (DFREC) to fill this gap. DFREC aims to recover the pair of source and target faces from a deepfake image to facilitate deepfake identity tracing and reduce the risk of deepfake attack. It comprises three key components: an Identity Segmentation Module (ISM), a Source Identity Reconstruction Module (SIRM), and a Target Identity Reconstruction Module (TIRM). The ISM segments the input face into distinct source and target face information, and the SIRM reconstructs the source face and extracts latent target identity features with the segmented source information. The background context and latent target identity features are synergetically fused by a Masked Autoencoder in the TIRM to reconstruct the target face. We evaluate DFREC on six different high-fidelity face-swapping attacks on FaceForensics++, CelebaMegaFS and FFHQ-E4S datasets, which demonstrate its superior recovery performance over state-of-the-art deepfake recovery algorithms. In addition, DFREC is the only scheme that can recover both pristine source and target faces directly from the forgery image with high fadelity.

Problem

• Recover source and target faces from deepfake images
• Enhance deepfake identity tracing and forensic investigation
• Improve interpretability and identity traceability in deepfake detection

Innovation

• Identity-aware Masked Autoencoder for DeepFake recovery
• Segments and reconstructs source and target faces
• Enhances identity tracing and forensic investigation accuracy