Towards Reliable Identification of Diffusion-based Image Manipulations

2025-06-05

AI Summary

Detecting and localizing semantic manipulations—especially in facial expressions, gestures, and background regions—remains challenging for diffusion-based image forgery detection due to poor generalization across unseen models. To address this, we propose RADAR, a robust detection and localization framework that leverages foundation models to extract multimodal features, integrates a localizing network with auxiliary contrastive loss, and achieves high-precision manipulation detection and pixel-level localization. We introduce BBC-PAIR, a novel benchmark comprising 28 state-of-the-art diffusion models, enabling rigorous evaluation of cross-model generalization for both seen and unseen generators. Extensive experiments demonstrate that RADAR significantly outperforms existing SOTA methods on BBC-PAIR, achieving superior detection accuracy and fine-grained localization precision. Our approach establishes a scalable, robust paradigm for generative image forensics, advancing both generalizability and interpretability in diffusion model attribution.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Changing facial expressions, gestures, or background details may dramatically alter the meaning conveyed by an image. Notably, recent advances in diffusion models greatly improve the quality of image manipulation while also opening the door to misuse. Identifying changes made to authentic images, thus, becomes an important task, constantly challenged by new diffusion-based editing tools. To this end, we propose a novel approach for ReliAble iDentification of inpainted AReas (RADAR). RADAR builds on existing foundation models and combines features from different image modalities. It also incorporates an auxiliary contrastive loss that helps to isolate manipulated image patches. We demonstrate these techniques to significantly improve both the accuracy of our method and its generalisation to a large number of diffusion models. To support realistic evaluation, we further introduce BBC-PAIR, a new comprehensive benchmark, with images tampered by 28 diffusion models. Our experiments show that RADAR achieves excellent results, outperforming the state-of-the-art in detecting and localising image edits made by both seen and unseen diffusion models. Our code, data and models will be publicly available at alex-costanzino.github.io/radar.

Problem

• Detect manipulated areas in images altered by diffusion models
• Improve accuracy and generalization in identifying image edits
• Address misuse of advanced diffusion-based image manipulation tools

Innovation

• Combines features from different image modalities
• Uses auxiliary contrastive loss for isolation
• Introduces BBC-PAIR benchmark for evaluation