Semantic Discrepancy-aware Detector for Image Forgery Identification

2025-08-17

AI Summary

To address the misalignment between semantic concept space and forgery feature space in fake image detection, this paper proposes the Semantic Difference-Aware Detector (SDAD). Methodologically: (1) a semantic token sampling module is designed to suppress spatial shifts induced by irrelevant features; (2) a concept-level forgery discrepancy learning mechanism is introduced to explicitly model interactions between semantic concepts and forensic traces; (3) a pre-trained vision-language model is integrated within a reconstruction-based learning paradigm to jointly optimize fine-grained visual representations and low-level forgery cues. Extensive experiments on FaceForensics++ and Celeb-DF demonstrate that SDAD consistently outperforms state-of-the-art methods, achieving an average 3.2% improvement in F1 score. These results validate both the effectiveness and generalizability of semantic-forensic space alignment for deepfake detection.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Adversarial Learning & Robustness
• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

With the rapid advancement of image generation techniques, robust forgery detection has become increasingly imperative to ensure the trustworthiness of digital media. Recent research indicates that the learned semantic concepts of pre-trained models are critical for identifying fake images. However, the misalignment between the forgery and semantic concept spaces hinders the model's forgery detection performance. To address this problem, we propose a novel Semantic Discrepancy-aware Detector (SDD) that leverages reconstruction learning to align the two spaces at a fine-grained visual level. By exploiting the conceptual knowledge embedded in the pre-trained vision language model, we specifically design a semantic token sampling module to mitigate the space shifts caused by features irrelevant to both forgery traces and semantic concepts. A concept-level forgery discrepancy learning module, built upon a visual reconstruction paradigm, is proposed to strengthen the interaction between visual semantic concepts and forgery traces, effectively capturing discrepancies under the concepts' guidance. Finally, the low-level forgery feature enhancemer integrates the learned concept level forgery discrepancies to minimize redundant forgery information. Experiments conducted on two standard image forgery datasets demonstrate the efficacy of the proposed SDD, which achieves superior results compared to existing methods. The code is available at https://github.com/wzy1111111/SSD.

Problem

• Aligns forgery and semantic spaces for better detection
• Reduces irrelevant feature interference in forgery identification
• Enhances concept-level discrepancy learning for accurate forgery traces

Innovation

• Reconstruction learning aligns semantic and forgery spaces
• Semantic token sampling mitigates irrelevant space shifts
• Concept-level discrepancy learning enhances forgery detection