Diffusion models for Gaussian distributions: Exact solutions and Wasserstein errors

2024-05-23

arXiv.org

AI Summary

This work addresses the theoretical behavior and numerical error control of diffusion models under Gaussian data distributions. We systematically analyze the analytical solutions of the backward stochastic differential equation (SDE) and probability flow ordinary differential equation (ODE), and— for the first time—establish a rigorous, term-wise decomposition and exact quantification framework for four fundamental error sources: initialization, truncation, discretization, and score approximation, all measured in Wasserstein distance. Leveraging properties of Gaussian processes and SDE/ODE theory, we prove that all analytical solutions and mainstream discretization schemes remain Gaussian processes, enabling closed-form error computation directly in the data space. This yields the first complete error spectrum for diffusion sampling under Gaussian assumptions, eliminating reliance on proxy metrics (e.g., Inception Score) and permitting direct verification of sampler optimality. Our results provide a strict, computationally tractable theoretical benchmark for both analysis and algorithm design of diffusion models.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Web Mining and Content Analysis: Content-based information diffusion
• Security and Privacy: Data transparency and provenance

Abstract

Diffusion or score-based models recently showed high performance in image generation. They rely on a forward and a backward stochastic differential equations (SDE). The sampling of a data distribution is achieved by solving numerically the backward SDE or its associated flow ODE. Studying the convergence of these models necessitates to control four different types of error: the initialization error, the truncation error, the discretization and the score approximation. In this paper, we study theoretically the behavior of diffusion models and their numerical implementation when the data distribution is Gaussian. In this restricted framework where the score function is a linear operator, we derive the analytical solutions of the backward SDE and the probability flow ODE. We prove that these solutions and their discretizations are all Gaussian processes, which allows us to compute exact Wasserstein errors induced by each error type for any sampling scheme. Monitoring convergence directly in the data space instead of relying on Inception features, our experiments show that the recommended numerical schemes from the diffusion models literature are also the best sampling schemes for Gaussian distributions.

Problem

• Exact solutions for backward SDE and flow ODE in Gaussian diffusion models
• Wasserstein error analysis between target and sampled distributions
• Theoretical study of convergence errors in Gaussian data diffusion models

Innovation

• Analytical solutions for backward SDE and ODE
• Exact Wasserstein errors computation
• Gaussian process discretization analysis