TruthLens: Explainable DeepFake Detection for Face Manipulated and Fully Synthetic Data

2025-03-20

AI Summary

Existing DeepFake detection methods predominantly adopt black-box binary classification, lacking unified modeling capabilities for both facial manipulation and fully synthetic images, and failing to support region-level (e.g., eyes/nose/mouth) authenticity assessment or interpretable reasoning. Method: We propose the first fine-grained, interpretable detection framework for facial manipulation and full synthesis, innovatively integrating PaliGemma2—a multimodal large language model—with DINOv2—a self-supervised vision backbone—to jointly enable global semantic understanding and local texture modeling. Contribution/Results: Beyond binary classification, our framework generates human-readable, region-level textual explanations. Extensive experiments demonstrate consistent improvements of 2–14% in detection accuracy over state-of-the-art methods across cross-domain and cross-technique benchmarks, significantly enhancing generalization capability and interpretability.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)

Application Category

• Web Mining and Content Analysis: Large pretrained models with web data
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Detecting DeepFakes has become a crucial research area as the widespread use of AI image generators enables the effortless creation of face-manipulated and fully synthetic content, yet existing methods are often limited to binary classification (real vs. fake) and lack interpretability. To address these challenges, we propose TruthLens, a novel and highly generalizable framework for DeepFake detection that not only determines whether an image is real or fake but also provides detailed textual reasoning for its predictions. Unlike traditional methods, TruthLens effectively handles both face-manipulated DeepFakes and fully AI-generated content while addressing fine-grained queries such as"Does the eyes/nose/mouth look real or fake?"The architecture of TruthLens combines the global contextual understanding of multimodal large language models like PaliGemma2 with the localized feature extraction capabilities of vision-only models like DINOv2. This hybrid design leverages the complementary strengths of both models, enabling robust detection of subtle manipulations while maintaining interpretability. Extensive experiments on diverse datasets demonstrate that TruthLens outperforms state-of-the-art methods in detection accuracy (by 2-14%) and explainability, in both in-domain and cross-data settings, generalizing effectively across traditional and emerging manipulation techniques.

Problem

• Detects DeepFakes with detailed textual explanations
• Handles both face-manipulated and fully synthetic content
• Improves detection accuracy and interpretability over existing methods

Innovation

• Combines multimodal large language models with vision-only models
• Provides detailed textual reasoning for DeepFake predictions
• Handles both face-manipulated and fully synthetic content