Stochastic Gradient Descent over P2

2026-09-11

AI Summary

研究通过将问题提升到线性希尔伯特空间并构造高斯随机场近似，解决了在概率测度上使用随机梯度下降进行优化的问题。

Abstract

Stochastic gradient descent (SGD) admits diffusion approximations that replace the complicated randomness of stochastic gradients by Gaussian noise, providing a powerful tool for understanding its dynamics and long-time behavior. We investigate whether an analogous approximation principle holds for optimization over probability measures, where the objective is a functional defined on the Wasserstein space P2. The nonlinear geometry and infinite-dimensional nature of P2 prevent a direct extension of the classical Euclidean theory. Using Lions differentiability, we lift the problem to a linear Hilbert space, where higher-order differential calculus becomes available. We then construct a Gaussian random-field approximation whose velocity field matches the mean and covariance of the original stochastic gradient. By exploiting this moment matching through higher-order Taylor expansions, we show that the Gaussian approximation captures the SGD dynamics with second-order weak accuracy. Our result provides a rigorous foundation for replacing sample-driven randomness by analytically tractable Gaussian fluctuations in stochastic optimization over probability measures.

Problem

• Stochastic Gradient Descent
• Wasserstein Space P2
• Gaussian Approximation
• Lions Differentiability
• Higher-order Taylor Expansions

Innovation

• Stochastic Gradient Descent
• Wasserstein Space P2
• Gaussian Approximation
• Lions Differentiability
• Higher-order Taylor Expansions