On the computational cost of Stochastic Gradient Langevin Dynamics

2026-09-15

AI Summary

研究了SGLD与EM方法在不同参数下的计算成本，通过理论分析和数值实验比较了两种方法的优劣，特别是在大数据集和小批量情况下的表现。

Abstract

Stochastic Gradient Langevin Dynamics (SGLD) reduces the cost of Langevin-based sampling by replacing full-dataset drift evaluations with mini-batch approximations, but the resulting subsampling error may offset this computational saving. We study this trade-off for stochastic differential equations with finite-sum drifts and compare the computational cost of SGLD with that of the Euler-Maruyama (EM) method. For a prescribed mean-square accuracy $\varepsilon^2$, we derive complexity estimates that explicitly track the dependence on the dataset size $m$, mini-batch size $s$, and accuracy parameter $\varepsilon$. The resulting comparison reveals distinct parameter regimes in which either method is preferable. In particular, EM can have lower leading-order cost only in a small-data, aggressive-subsampling regime, whereas SGLD is favoured over most of the remaining parameter space. In the practically relevant regime $s \ll m$, the transition between the two methods occurs at the scale $m \asymp \varepsilon^{-1}$. We complement the theoretical analysis with numerical experiments based on a Gaussian Bayesian inference model, which examine the predicted cost regimes together with the underlying discretisation error and variance estimates.

Problem

• Stochastic Gradient Langevin Dynamics
• Euler-Maruyama method
• computational cost
• subsampling error
• finite-sum drifts

Innovation

• Stochastic Gradient Langevin Dynamics
• Euler-Maruyama method
• computational cost
• finite-sum drifts
• mean-square accuracy