Unlocking the Forgery Detection Potential of Vanilla MLLMs: A Novel Training-Free Pipeline

2025-11-17

AI Summary

Existing image forgery detection and localization (IFDL) methods suffer from poor generalization and weak interpretability; moreover, MLLM-based approaches typically rely on large-scale fine-tuning, failing to harness their intrinsic zero-shot generalization capability. Method: We propose Foresee—a training-free MLLM inference framework that synergistically activates the generic forgery recognition ability of vanilla MLLMs via a type-prior-driven prompting strategy and a flexible feature detector (FFD) specifically designed for copy-move manipulation. Contribution/Results: Without updating any model parameters, Foresee achieves high-precision forgery localization and natural-language explanations. It significantly outperforms state-of-the-art methods across diverse datasets and multiple forgery types—including unseen ones—demonstrating, for the first time, the strong generalization potential of pure inference-based MLLMs in IFDL tasks.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Diffusion Models for Vision
• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

With the rapid advancement of artificial intelligence-generated content (AIGC) technologies, including multimodal large language models (MLLMs) and diffusion models, image generation and manipulation have become remarkably effortless. Existing image forgery detection and localization (IFDL) methods often struggle to generalize across diverse datasets and offer limited interpretability. Nowadays, MLLMs demonstrate strong generalization potential across diverse vision-language tasks, and some studies introduce this capability to IFDL via large-scale training. However, such approaches cost considerable computational resources, while failing to reveal the inherent generalization potential of vanilla MLLMs to address this problem. Inspired by this observation, we propose Foresee, a training-free MLLM-based pipeline tailored for image forgery analysis. It eliminates the need for additional training and enables a lightweight inference process, while surpassing existing MLLM-based methods in both tamper localization accuracy and the richness of textual explanations. Foresee employs a type-prior-driven strategy and utilizes a Flexible Feature Detector (FFD) module to specifically handle copy-move manipulations, thereby effectively unleashing the potential of vanilla MLLMs in the forensic domain. Extensive experiments demonstrate that our approach simultaneously achieves superior localization accuracy and provides more comprehensive textual explanations. Moreover, Foresee exhibits stronger generalization capability, outperforming existing IFDL methods across various tampering types, including copy-move, splicing, removal, local enhancement, deepfake, and AIGC-based editing. The code will be released in the final version.

Problem

• Detecting image forgeries without training using vanilla MLLMs
• Improving generalization across diverse tampering types and datasets
• Enhancing both localization accuracy and interpretability of detections

Innovation

• Training-free pipeline for image forgery analysis
• Type-prior-driven strategy with Flexible Feature Detector
• Unleashes vanilla MLLMs potential without additional training