🤖 AI Summary
Existing methods have not explored the potential of multimodal large language models (M-LLMs) for image forgery detection; direct application often induces hallucination and overthinking, resulting in inaccurate reasoning and coarse-grained localization.
Method: We propose the first Chain-of-Clues prompting paradigm tailored for forgery detection, accompanied by theForgeryAnalysis dataset; design a joint clue-fusion and segmentation-generation architecture enabling text-guided pixel-level localization; and introduce a data synthesis and augmentation engine to support large-scale pretraining.
Contribution/Results: Our approach significantly improves generalizability, robustness, and interpretability. It outperforms state-of-the-art methods across multiple benchmarks and achieves, for the first time, end-to-end, interpretable, fine-grained M-LLM–driven forgery analysis.
📝 Abstract
Multimodal large language models have unlocked new possibilities for various multimodal tasks. However, their potential in image manipulation detection remains unexplored. When directly applied to the IMD task, M-LLMs often produce reasoning texts that suffer from hallucinations and overthinking. To address this, in this work, we propose ForgerySleuth, which leverages M-LLMs to perform comprehensive clue fusion and generate segmentation outputs indicating specific regions that are tampered with. Moreover, we construct the ForgeryAnalysis dataset through the Chain-of-Clues prompt, which includes analysis and reasoning text to upgrade the image manipulation detection task. A data engine is also introduced to build a larger-scale dataset for the pre-training phase. Our extensive experiments demonstrate the effectiveness of ForgeryAnalysis and show that ForgerySleuth significantly outperforms existing methods in generalization, robustness, and explainability.