FakeReasoning: Towards Generalizable Forgery Detection and Reasoning

2025-03-27

AI Summary

Detecting AI-generated images across diverse generative models remains challenging, and existing saliency-based explanation methods fail to provide reliable interpretability. Method: This paper introduces the Forgery Detection and Reasoning (FDR) task and constructs MMFR-Dataset—the first large-scale (100K+) multimodal forgery reasoning benchmark—featuring ten fine-grained forgery attribution categories. We propose a novel forgery-aligned contrastive learning framework coupled with classification probability calibration mapping, integrating vision-language models (VLMs) with both cross-modal and intra-modal contrastive learning to achieve semantic alignment of forgery attributes and structured textual attribution. Contribution/Results: Our approach significantly outperforms state-of-the-art methods across multiple generative models, improving average detection accuracy by 5.2%. Crucially, it simultaneously delivers high-fidelity, verifiable natural-language reasoning outputs—uniquely unifying robust forgery detection and interpretable, attributable reasoning in a single framework.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Large Vision Models
• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems

Abstract

Accurate and interpretable detection of AI-generated images is essential for mitigating risks associated with AI misuse. However, the substantial domain gap among generative models makes it challenging to develop a generalizable forgery detection model. Moreover, since every pixel in an AI-generated image is synthesized, traditional saliency-based forgery explanation methods are not well suited for this task. To address these challenges, we propose modeling AI-generated image detection and explanation as a Forgery Detection and Reasoning task (FDR-Task), leveraging vision-language models (VLMs) to provide accurate detection through structured and reliable reasoning over forgery attributes. To facilitate this task, we introduce the Multi-Modal Forgery Reasoning dataset (MMFR-Dataset), a large-scale dataset containing 100K images across 10 generative models, with 10 types of forgery reasoning annotations, enabling comprehensive evaluation of FDR-Task. Additionally, we propose FakeReasoning, a forgery detection and reasoning framework with two key components. First, Forgery-Aligned Contrastive Learning enhances VLMs' understanding of forgery-related semantics through both cross-modal and intra-modal contrastive learning between images and forgery attribute reasoning. Second, a Classification Probability Mapper bridges the optimization gap between forgery detection and language modeling by mapping the output logits of VLMs to calibrated binary classification probabilities. Experiments across multiple generative models demonstrate that FakeReasoning not only achieves robust generalization but also outperforms state-of-the-art methods on both detection and reasoning tasks.

Problem

• Detecting AI-generated images accurately and interpretably
• Bridging domain gaps in generalizable forgery detection
• Providing structured reasoning for forgery attributes using VLMs

Innovation

• Leverages vision-language models for forgery detection
• Introduces Multi-Modal Forgery Reasoning dataset
• Uses Forgery-Aligned Contrastive Learning for semantics