HFI: A unified framework for training-free detection and implicit watermarking of latent diffusion model generated images

2024-12-30

AI Summary

The rapid evolution of latent diffusion models (LDMs) has exacerbated challenges in AI-generated image forensics, while existing training-free detection methods fail on simple-background images and lack model provenance capability. Method: We propose a training-free, universal detection and implicit watermarking framework. Its core innovation lies in modeling the LDM autoencoder as a frequency-domain sampling kernel, leveraging its intrinsic architectural properties to quantify aliasing distortion induced during reconstruction—enabling background-robust, unsupervised discrimination. The method jointly exploits frequency-domain aliasing metrics and unsupervised reconstruction distortion estimation, requiring no real/forged image labels or training data. Results: Extensive experiments demonstrate that our approach significantly outperforms state-of-the-art training-free baselines across diverse LDM-generated images. Moreover, it enables high-accuracy model attribution—functioning as a model-specific implicit watermark—while maintaining computational efficiency and strong generalization across unseen architectures and generation settings.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Dramatic advances in the quality of the latent diffusion models (LDMs) also led to the malicious use of AI-generated images. While current AI-generated image detection methods assume the availability of real/AI-generated images for training, this is practically limited given the vast expressibility of LDMs. This motivates the training-free detection setup where no related data are available in advance. The existing LDM-generated image detection method assumes that images generated by LDM are easier to reconstruct using an autoencoder than real images. However, we observe that this reconstruction distance is overfitted to background information, leading the current method to underperform in detecting images with simple backgrounds. To address this, we propose a novel method called HFI. Specifically, by viewing the autoencoder of LDM as a downsampling-upsampling kernel, HFI measures the extent of aliasing, a distortion of high-frequency information that appears in the reconstructed image. HFI is training-free, efficient, and consistently outperforms other training-free methods in detecting challenging images generated by various generative models. We also show that HFI can successfully detect the images generated from the specified LDM as a means of implicit watermarking. HFI outperforms the best baseline method while achieving magnitudes of

Problem

• Image Generation
• Forgery Detection
• Adversarial Adaptation

Innovation

• HFI Method
• Alias Analysis
• Untrained Image Detection