Towards a Mechanistic Explanation of Diffusion Model Generalization

2024-11-28

arXiv.org

AI Summary

This paper investigates the intrinsic mechanisms underlying the generalization capability of diffusion models, particularly their robust cross-architecture performance. Method: We propose a training-free, interpretable analytical framework that establishes, for the first time, a theoretical connection between diffusion model generalization and local empirical denoisers. Our approach models local inductive biases, designs a multi-scale local denoiser aggregation algorithm, and evaluates behavioral consistency across forward and reverse processes. Contribution/Results: We demonstrate that generalization primarily arises from local denoising operations providing a high-fidelity approximation to the training objective. Experiments show our method achieves superior visual fidelity and lower mean squared error in reproducing neural denoising behavior compared to existing approaches. Furthermore, we validate the universality of the “local denoising dominates generalization” principle across diverse network architectures, thereby overcoming the black-box attribution bottleneck in diffusion model analysis.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization

Abstract

We propose a simple, training-free mechanism which explains the generalization behaviour of diffusion models. By comparing pre-trained diffusion models to their theoretically optimal empirical counterparts, we identify a shared local inductive bias across a variety of network architectures. From this observation, we hypothesize that network denoisers generalize through localized denoising operations, as these operations approximate the training objective well over much of the training distribution. To validate our hypothesis, we introduce novel denoising algorithms which aggregate local empirical denoisers to replicate network behaviour. Comparing these algorithms to network denoisers across forward and reverse diffusion processes, our approach exhibits consistent visual similarity to neural network outputs, with lower mean squared error than previously proposed methods.

Problem

• Explain diffusion model generalization
• Identify local inductive bias
• Develop novel denoising algorithms

Innovation

• Training-free mechanism explains generalization
• Local inductive bias identified across architectures
• Novel denoising algorithms replicate network behavior