Local Coverage Governs Memorization in Diffusion Models

2026-06-12

AI Summary

This work uncovers the local mechanism underlying memorization in diffusion models—specifically, how a single model can simultaneously generate both memorized and novel samples. Building on the theoretical connection between diffusion processes and kernel density estimation, the study proposes that memorization is governed by the local coverage density of training data: regions with low coverage tend to memorize isolated samples, whereas densely covered regions facilitate generalization. The paper establishes the first theoretical link between memorization and the local geometric structure of data, introducing a “local coverage criterion” to predict where memorization occurs. It further elucidates how intra-class sparsity in multi-class settings modulates memorization strength. Experiments confirm that memorization intensifies with increasing local sparsity and demonstrate the coexistence of memorized and novel generation within the same model.

Abstract

Memorization in diffusion models is often treated as a global property of the model or dataset. In practice, however, a single diffusion model can simultaneously generate both memorized and novel samples. Which training samples are most likely to be memorized? In this work, we show that memorization is governed by \emph{local data coverage}. Leveraging the connection between diffusion models and kernel density estimation (KDE), we derive a theoretical criterion that predicts whether a point is memorized based on the density of training data in its neighborhood and the size of the training dataset. In the high-dimensional limit, this leads to a sharp, local transition: regions of low coverage are dominated by isolated training samples, which are memorized, while dense regions support interpolation and generalization. We validate these predictions empirically, showing that memorization increases with local sparsity and that diffusion models exhibit a coexistence of memorized and novel samples within the same model. Extending this framework to multi-class settings, we further show that classes with higher intra-class sparsity (and thus lower local coverage) are more strongly memorized. Our results provide a local view of memorization in diffusion models, explaining when and where memorization occurs in terms of data geometry.

Problem

• memorization
• diffusion models
• local coverage
• data sparsity
• generalization

Innovation

• local coverage
• memorization
• diffusion models
• kernel density estimation
• data sparsity