Diffusion Path Samplers via Sequential Monte Carlo

2026-01-29

AI Summary

This work addresses the lack of efficient and theoretically grounded sampling methods for target distributions known only up to an unnormalized density. We propose a diffusion-path-based sequential Monte Carlo (SMC) framework that bridges a simple base distribution and the target distribution through a diffusion process. By integrating diffusion-annealed Langevin dynamics and introducing SMC-evolved auxiliary variables to accurately estimate time-dependent score functions, our approach enables more precise sampling. Furthermore, we design a novel control variate scheduling strategy to effectively reduce estimation variance. Theoretical analysis establishes convergence guarantees for the algorithm, and experiments on multiple synthetic and real-world datasets demonstrate substantial improvements in both sampling efficiency and estimation accuracy.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Content-based information diffusion

Abstract

We develop a diffusion-based sampler for target distributions known up to a normalising constant. To this end, we rely on the well-known diffusion path that smoothly interpolates between a (simple) base distribution and the target distribution, widely used in diffusion models. Our approach is based on a practical implementation of diffusion-annealed Langevin Monte Carlo, which approximates the diffusion path with convergence guarantees. We tackle the score estimation problem by developing an efficient sequential Monte Carlo sampler that evolves auxiliary variables from conditional distributions along the path, which provides principled score estimates for time-varying distributions. We further develop novel control variate schedules that minimise the variance of these score estimates. Finally, we provide theoretical guarantees and empirically demonstrate the effectiveness of our method on several synthetic and real-world datasets.

Problem

• diffusion models
• score estimation
• Monte Carlo sampling
• unnormalized distributions
• sequential Monte Carlo

Innovation

• Sequential Monte Carlo
• Diffusion Models
• Score Estimation
• Control Variates
• Langevin Monte Carlo