Underdamped Diffusion Bridges with Applications to Sampling

2025-03-02

AI Summary

This work addresses the challenge of efficient sampling from unnormalized densities without access to target samples. Methodologically, it introduces the Underdamped Diffusion Bridge (UDB) framework: (i) it establishes, for the first time under underdamped stochastic differential equations (SDEs), the rigorous equivalence between score matching and maximizing a variational lower bound; (ii) it abandons fixed noise schedules and instead learns a flexible, task-agnostic density evolution path; and (iii) it incorporates a degenerate diffusion matrix and high-order numerical integrators to enhance stability and accuracy. The key contributions are: (i) an end-to-end sampling scheme from a prior to an unnormalized target distribution—requiring neither target samples nor hyperparameter tuning; and (ii) state-of-the-art performance across diverse no-sample tasks, with significantly reduced sampling steps, while maintaining theoretical rigor and practical deployability.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Reasoning under Uncertainty: Stochastic Optimization
• Machine Learning: Learning with Manifolds

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

We provide a general framework for learning diffusion bridges that transport prior to target distributions. It includes existing diffusion models for generative modeling, but also underdamped versions with degenerate diffusion matrices, where the noise only acts in certain dimensions. Extending previous findings, our framework allows to rigorously show that score matching in the underdamped case is indeed equivalent to maximizing a lower bound on the likelihood. Motivated by superior convergence properties and compatibility with sophisticated numerical integration schemes of underdamped stochastic processes, we propose emph{underdamped diffusion bridges}, where a general density evolution is learned rather than prescribed by a fixed noising process. We apply our method to the challenging task of sampling from unnormalized densities without access to samples from the target distribution. Across a diverse range of sampling problems, our approach demonstrates state-of-the-art performance, notably outperforming alternative methods, while requiring significantly fewer discretization steps and no hyperparameter tuning.

Problem

• Learning diffusion bridges for sampling from unnormalized densities
• Underdamped diffusion bridges with degenerate diffusion matrices
• State-of-the-art performance in sampling with fewer steps

Innovation

• Underdamped diffusion bridges for density evolution
• Score matching equivalent to likelihood maximization
• State-of-the-art sampling with fewer discretization steps