Sequential Controlled Langevin Diffusions

2024-12-10

International Conference on Learning Representations

AI Summary

To address the slow convergence of sequential Monte Carlo (SMC) and training instability in diffusion-based sampling for unnormalized densities, this paper introduces, for the first time, a unified continuous-time path-space framework that jointly models SMC and learned diffusion processes via controllable Langevin dynamics. The framework embeds transport and resampling operations coherently into a stochastic differential equation (SDE) path manifold, enabling adaptive state transitions and robust importance reweighting through joint optimization of path-space measures and gradient-guided updates. It inherits the asymptotic guarantees of SMC while retaining the expressive power of diffusion models. Empirically, the method significantly outperforms existing samplers across multiple benchmarks: convergence accelerates by 2–5×; training cost is reduced to only 10% of comparable diffusion samplers; and it achieves strong theoretical stability alongside high sampling fidelity.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Machine Learning: Learning with Manifolds
• Computer Vision: Diffusion Models for Vision

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

An effective approach for sampling from unnormalized densities is based on the idea of gradually transporting samples from an easy prior to the complicated target distribution. Two popular methods are (1) Sequential Monte Carlo (SMC), where the transport is performed through successive annealed densities via prescribed Markov chains and resampling steps, and (2) recently developed diffusion-based sampling methods, where a learned dynamical transport is used. Despite the common goal, both approaches have different, often complementary, advantages and drawbacks. The resampling steps in SMC allow focusing on promising regions of the space, often leading to robust performance. While the algorithm enjoys asymptotic guarantees, the lack of flexible, learnable transitions can lead to slow convergence. On the other hand, diffusion-based samplers are learned and can potentially better adapt themselves to the target at hand, yet often suffer from training instabilities. In this work, we present a principled framework for combining SMC with diffusion-based samplers by viewing both methods in continuous time and considering measures on path space. This culminates in the new Sequential Controlled Langevin Diffusion (SCLD) sampling method, which is able to utilize the benefits of both methods and reaches improved performance on multiple benchmark problems, in many cases using only 10% of the training budget of previous diffusion-based samplers.

Problem

• Combining SMC and diffusion samplers for improved sampling
• Addressing slow convergence in SMC with flexible learned transitions
• Mitigating training instabilities in diffusion-based sampling methods

Innovation

• Combines SMC with diffusion-based samplers
• Uses controlled Langevin diffusion in path space
• Achieves improved performance with reduced training budget