Understanding Why Foundation Models Work for Diffusion-Generated Image Detection

2026-08-12

AI Summary

This study investigates why vision foundation models effectively detect images generated by diffusion models. Through DDIM inversion, frequency-domain component swapping, and latent space statistical analysis, the work systematically demonstrates that detectors primarily rely on non-semantic distributional biases in low-to-mid frequency bands rather than high-frequency artifacts. It further reveals that regenerated images exhibit reduced variance and lower effective dimensionality in the latent space. For the first time, this research elucidates the frequency-domain origins of foundation model–based detection mechanisms, establishing that their robustness stems from sensitivity to statistical anomalies in the low-to-mid frequencies of synthetic images. These findings provide a novel perspective and methodological foundation for interpretable image forensics.

Abstract

Vision foundation models have recently emerged as powerful feature extractors for detecting AI-generated images, achieving strong generalization across generators and robustness to common image degradations. However, the reason behind their effectiveness is poorly understood. In this work, we investigate what cues are exploited by foundation-model-based detectors to distinguish real images from diffusion-generated ones. To this end, we design an ad hoc analysis protocol based on DDIM inversion. Given a real image we generate a sequence of synthetic copies by changing the depth of DDIM inversion. Even though most copies are semantically identical to the real reference, the detector score varies significantly across them due to subtle traces introduced by the diffusion synthesis, showing that its decision is not primarily driven by semantic failures. Through a frequency-swapping analysis, we further reveal that the discriminative cues exploited by the detectors are mainly localized in the low-to-mid frequency range, rather than only in the high-frequency range, as is the case for artifacts commonly associated with generative models. Finally, a latent-space analysis shows that regenerated images exhibit reduced variance and effective dimensionality, indicating that diffusion models do not fully reproduce the variability of real data. Overall, our results suggest that foundation-model-based detectors succeed by capturing non-semantic low-to-mid frequency distributional discrepancies between real and diffusion-generated images. These findings provide new insight into the robustness and generalization of such detectors and suggest directions for more interpretable forensic methods.

Problem

• foundation models
• diffusion-generated image detection
• frequency analysis
• distributional discrepancies
• image forensics

Innovation

• foundation models
• diffusion-generated image detection
• frequency analysis
• DDIM inversion
• latent space