A Tutorial on Diffusion Theory: From Differential Equations to Diffusion Models

2026-05-21

AI Summary

This work establishes a unified theoretical framework for diffusion models from the perspective of differential equations. Starting from a conditional Gaussian forward process, it derives the corresponding forward stochastic differential equation (SDE) and ordinary differential equation (ODE), and constructs a dynamical system that transports the data distribution to a standard Gaussian prior via marginalization. The framework then introduces a reverse SDE and a probability flow ODE, both driven by the marginal score function, thereby unifying score matching and noise prediction objectives. It rigorously demonstrates the equivalence of DDPM and DDIM in their training objectives while clarifying their fundamental distinction in sampling mechanisms—DDPM corresponds to a discretized reverse SDE, whereas DDIM implements a reverse ODE. Furthermore, the framework seamlessly incorporates mainstream sampling techniques such as DPM-Solver and classifier guidance, providing a coherent and rigorous continuous-time foundation for diffusion models.

Abstract

This tutorial develops diffusion models from the viewpoint of differential equations. We begin with the conditional Gaussian forward process and show that this path admits both an ordinary differential equation (ODE) representation and a stochastic differential equation (SDE) representation. Averaging the conditional process over the data distribution then yields marginalized forward ODE and SDE formulations that transport the data distribution $p_0=p_{\mathrm{data}}$ to a Gaussian prior $p_1=\mathcal{N}(0,I)$. We next derive the corresponding reverse-time dynamics, namely the reverse SDE and the reverse probability-flow ODE, both of which are governed by the marginal score $\grad\log p_t(x)$. This leads to a training objective for score estimation and shows that the standard noise-prediction objective is equivalent to score matching up to an additive constant independent of the model parameters. We then discuss sampling methods for the learned reverse dynamics, including DPM-Solver, as well as guided sampling through classifier guidance and classifier-free guidance. Finally, we compare DDPM and DDIM with the reverse SDE/ODE framework and show that they share the same training objective, while DDPM sampling corresponds to discrete reverse-SDE sampling and DDIM sampling corresponds to reverse-ODE sampling.

Problem

• diffusion models
• differential equations
• score matching
• SDE
• ODE

Innovation

• diffusion models
• stochastic differential equations
• score matching
• reverse dynamics
• ODE/SDE framework