Spacetime Geometry of Denoising in Diffusion Models

2025-05-23

AI Summary

Diffusion model denoising is typically treated as a black-box sampling process lacking geometric interpretability and physical meaning. Method: This work models the denoising process from an information-geometric perspective, viewing the family of distributions at varying noise levels as a statistical manifold parameterized by noise intensity. It establishes, for the first time, a Fisher–Rao metric structure over an exponential family, wherein denoising trajectories correspond naturally to geodesics on this manifold. Leveraging efficient numerical geodesic solvers, the method enables rapid sampling of continuous, energy-smooth transition paths in high-dimensional spaces, yielding Boltzmann-distributed pathways connecting metastable states. Contribution/Results: The core advance lies in uncovering the intrinsic geometric structure of diffusion denoising—transforming it from an opaque sampling procedure into a principled, interpretable, and computationally tractable geodesic problem. The approach requires no retraining and yields physically meaningful, smooth latent trajectories. Code is publicly available.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Learning with Manifolds
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Semantics and Knowledge: Methods, algorithms and applications for the development of semantic models, knowledge graphs and other forms of structured data models with machine-interpretable semantics

Abstract

We present a novel perspective on diffusion models using the framework of information geometry. We show that the set of noisy samples, taken across all noise levels simultaneously, forms a statistical manifold -- a family of denoising probability distributions. Interpreting the noise level as a temporal parameter, we refer to this manifold as spacetime. This manifold naturally carries a Fisher-Rao metric, which defines geodesics -- shortest paths between noisy points. Notably, this family of distributions is exponential, enabling efficient geodesic computation even in high-dimensional settings without retraining or fine-tuning. We demonstrate the practical value of this geometric viewpoint in transition path sampling, where spacetime geodesics define smooth sequences of Boltzmann distributions, enabling the generation of continuous trajectories between low-energy metastable states. Code is available at: https://github.com/Aalto-QuML/diffusion-spacetime-geometry.

Problem

• Modeling noisy samples as statistical manifold in diffusion models
• Computing geodesics efficiently in high-dimensional exponential families
• Generating continuous trajectories between metastable states via spacetime geometry

Innovation

• Information geometry framework for diffusion models
• Fisher-Rao metric enables efficient geodesic computation
• Spacetime geodesics sample Boltzmann distributions smoothly