Diffusion Noise Feature: Accurate and Fast Generated Image Detection

2023-12-05

arXiv.org

AI Summary

Image generation detection suffers from low accuracy and poor generalization across models and datasets. To address this, we propose Diffusion Noise Features (DNF)—the first method to identify and exploit a semantic gap in the implicit Gaussian noise distribution between real and synthetic images during the reverse denoising process of pretrained diffusion models (e.g., Stable Diffusion). DNF extracts lightweight, universal, and fine-tuning-free discriminative noise representations. These features can be directly fed into lightweight classifiers (e.g., ResNet-50) for end-to-end detection. Evaluated on four training datasets and five unseen test datasets—including out-of-distribution samples and unknown generators—DNF achieves state-of-the-art performance, maintaining high accuracy and strong robustness without retraining. It significantly enhances cross-model and cross-dataset generalization, setting a new benchmark for scalable, model-agnostic synthetic image detection.

Abstract

Generative models have reached an advanced stage where they can produce remarkably realistic images. However, this remarkable generative capability also introduces the risk of disseminating false or misleading information. Notably, existing image detectors for generated images encounter challenges such as low accuracy and limited generalization. This paper seeks to address this issue by seeking a representation with strong generalization capabilities to enhance the detection of generated images. Our investigation has revealed that real and generated images display distinct latent Gaussian representations when subjected to an inverse diffusion process within a pre-trained diffusion model. Exploiting this disparity, we can amplify subtle artifacts in generated images. Building upon this insight, we introduce a novel image representation known as Diffusion Noise Feature (DNF). DNF is extracted from the estimated noise generated during the inverse diffusion process. A simple classifier, e.g., ResNet50, trained on DNF achieves high accuracy, robustness, and generalization capabilities for detecting generated images (even the corresponding generator is built with datasets/structures that are not seen during the classifier's training). We conducted experiments using four training datasets and five testsets, achieving state-of-the-art detection performance.

Problem

• Detecting AI-generated images with high accuracy
• Improving generalization across unseen generators and content
• Addressing low accuracy in current detection methods

Innovation

• Extracts diffusion noise features as fingerprints
• Trains ResNet-50 classifier on amplified artifacts
• Detects unseen generators with high generalization