The Picard-Lagrange Framework for Higher-Order Langevin Monte Carlo

2025-10-20

AI Summary

Efficient sampling from log-concave distributions remains challenging, especially for high-accuracy requirements. Method: We propose the first scalable high-order Langevin Monte Carlo framework applicable to arbitrary order $K geq 3$. It is grounded in $K$-th-order Langevin dynamics, employs Lagrange interpolation to approximate the gradient of the potential, and uses Picard iteration to correct interpolation errors—yielding a high-fidelity discretization. Contribution/Results: Under smoothness and strong log-concavity assumptions, we establish a Wasserstein-2 convergence guarantee with gradient query complexity $ ilde{O}(d^{(K-1)/(2K-3)} varepsilon^{-2/(2K-3)})$, strictly improving upon all existing first- through third-order methods and monotonically improving with $K$. This is the first work to derive an optimal gradient complexity bound for high-order Langevin sampling, introducing a new paradigm for discretizing high-order stochastic differential equations and accelerating sampling.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Machine Learning: Learning with Manifolds
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Sampling from log-concave distributions is a central problem in statistics and machine learning. Prior work establishes theoretical guarantees for Langevin Monte Carlo algorithm based on overdamped and underdamped Langevin dynamics and, more recently, some third-order variants. In this paper, we introduce a new sampling algorithm built on a general $K$th-order Langevin dynamics, extending beyond second- and third-order methods. To discretize the $K$th-order dynamics, we approximate the drift induced by the potential via Lagrange interpolation and refine the node values at the interpolation points using Picard-iteration corrections, yielding a flexible scheme that fully utilizes the acceleration of higher-order Langevin dynamics. For targets with smooth, strongly log-concave densities, we prove dimension-dependent convergence in Wasserstein distance: the sampler achieves $varepsilon$-accuracy within $widetilde O(d^{frac{K-1}{2K-3}}varepsilon^{-frac{2}{2K-3}})$ gradient evaluations for $K ge 3$. To our best knowledge, this is the first sampling algorithm achieving such query complexity. The rate improves with the order $K$ increases, yielding better rates than existing first to third-order approaches.

Problem

• Develops higher-order Langevin Monte Carlo for log-concave sampling
• Extends sampling beyond second- and third-order Langevin dynamics
• Achieves improved convergence rates with increasing order K

Innovation

• General Kth-order Langevin dynamics for sampling
• Lagrange interpolation with Picard-iteration corrections
• Dimension-dependent convergence with improved query complexity