X-Edit: Detecting and Localizing Edits in Images Altered by Text-Guided Diffusion Models

2025-05-16

AI Summary

This work addresses the detection of subtle, text-guided image manipulations generated by diffusion models—a critical challenge in AI-generated content forensics. We propose the first localization-aware deepfake detection method capable of precisely identifying tampered regions. Our approach leverages pre-trained diffusion models for image inversion to extract editing-sensitive features, and introduces an attention-driven semantic segmentation network integrating multi-scale reconstruction with a low-frequency-prior supervision strategy. A novel dual-objective loss function jointly optimizes pixel-level segmentation accuracy and low-frequency perceptual correlation. Key contributions include: (1) the first localized detection framework specifically designed for diffusion-based edits; (2) the first benchmark dataset comprising paired original–edited images; and (3) state-of-the-art performance—significantly surpassing existing baselines in PSNR and SSIM—while achieving high-precision pixel-level localization of fine-grained edits, demonstrating strong efficacy and robustness for forensic analysis of AI-generated imagery.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)

Application Category

• Web Mining and Content Analysis: Large pretrained models with web data
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

Text-guided diffusion models have significantly advanced image editing, enabling highly realistic and local modifications based on textual prompts. While these developments expand creative possibilities, their malicious use poses substantial challenges for detection of such subtle deepfake edits. To this end, we introduce Explain Edit (X-Edit), a novel method for localizing diffusion-based edits in images. To localize the edits for an image, we invert the image using a pretrained diffusion model, then use these inverted features as input to a segmentation network that explicitly predicts the edited masked regions via channel and spatial attention. Further, we finetune the model using a combined segmentation and relevance loss. The segmentation loss ensures accurate mask prediction by balancing pixel-wise errors and perceptual similarity, while the relevance loss guides the model to focus on low-frequency regions and mitigate high-frequency artifacts, enhancing the localization of subtle edits. To the best of our knowledge, we are the first to address and model the problem of localizing diffusion-based modified regions in images. We additionally contribute a new dataset of paired original and edited images addressing the current lack of resources for this task. Experimental results demonstrate that X-Edit accurately localizes edits in images altered by text-guided diffusion models, outperforming baselines in PSNR and SSIM metrics. This highlights X-Edit's potential as a robust forensic tool for detecting and pinpointing manipulations introduced by advanced image editing techniques.

Problem

• Detecting subtle deepfake edits in images
• Localizing diffusion-based modified image regions
• Addressing lack of datasets for edit localization

Innovation

• Inverts images using pretrained diffusion model
• Uses segmentation network with attention mechanisms
• Combines segmentation and relevance loss functions