Detecting and Mitigating Memorization in Diffusion Models through Anisotropy of the Log-Probability

2026-01-28

AI Summary

Diffusion models, while capable of generating high-fidelity images, may inadvertently memorize and reproduce training data, posing privacy and copyright risks. This work reveals for the first time that, under low-noise conditions, the guidance vectors of memorized samples exhibit significant angular alignment with the unconditional score. Leveraging this insight, we propose a denoising-free memorization detection metric that efficiently identifies memorized samples from pure noise inputs by integrating the isotropic norm of the log-probability distribution with its anisotropic directional information. Our method substantially outperforms existing denoising-free approaches on Stable Diffusion v1.4 and v2, achieving over a fivefold speedup in detection and enabling an adaptive prompt-based strategy for mitigating memorization.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Safety and Robustness

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Security and Privacy: Data transparency and provenance
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

Diffusion-based image generative models produce high-fidelity images through iterative denoising but remain vulnerable to memorization, where they unintentionally reproduce exact copies or parts of training images. Recent memorization detection methods are primarily based on the norm of score difference as indicators of memorization. We prove that such norm-based metrics are mainly effective under the assumption of isotropic log-probability distributions, which generally holds at high or medium noise levels. In contrast, analyzing the anisotropic regime reveals that memorized samples exhibit strong angular alignment between the guidance vector and unconditional scores in the low-noise setting. Through these insights, we develop a memorization detection metric by integrating isotropic norm and anisotropic alignment. Our detection metric can be computed directly on pure noise inputs via two conditional and unconditional forward passes, eliminating the need for costly denoising steps. Detection experiments on Stable Diffusion v1.4 and v2 show that our metric outperforms existing denoising-free detection methods while being at least approximately 5x faster than the previous best approach. Finally, we demonstrate the effectiveness of our approach by utilizing a mitigation strategy that adapts memorized prompts based on our developed metric. The code is available at https://github.com/rohanasthana/memorization-anisotropy .

Problem

• memorization
• diffusion models
• anisotropy
• log-probability
• privacy

Innovation

• anisotropy
• memorization detection
• diffusion models
• score alignment
• denoising-free