High-Probability PL-SGD with Markovian Noise: Optimal Mixing and Tail Dependence

2026-06-24

AI Summary

This work addresses smooth optimization under Markovian noise satisfying the Polyak–Łojasiewicz (PL) condition, bridging the theoretical gap between high-probability and expected convergence bounds caused by dependence on the mixing time. By employing a lagged blocking strategy for light-tailed noise and introducing a full-batch clipped blocking method for heavy-tailed noise, the authors leverage geometric mixing assumptions and finite $p$-th moment conditions to establish the first tight high-probability bound of $\widetilde{O}(t_{\text{mix}}/k)$ with linear dependence on the mixing time in the light-tailed setting, and prove its optimality. For heavy-tailed noise, they develop a robust optimization framework yielding a high-probability error bound of $\widetilde{O}\big(\sigma_p^2 (t_{\text{mix}}/T)^{2(p-1)/p}\big)$, accompanied by a matching lower bound.

Abstract

We study first-order methods for smooth objectives satisfying the Polyak-Łojasiewicz (PL) condition when gradient samples are generated by an exogenous Markov chain. In the light-tailed setting, prior uniform-in-time high-probability bounds for ordinary Stochastic Gradient Descent (SGD) under a standard growth envelope scale as $\widetilde{O}(t_{mix}^2/k)$, leaving a gap with the $\widetilde{O}(t_{mix}/k)$ expectation bounds. We close this gap using a lag-blocking argument to establish a uniform high-probability guarantee with a leading stochastic term of $\widetilde{O}(t_{mix}/(k+K_0))$ under geometric mixing. We prove this linear dependence on the mixing time is optimal via a matching $Ω(σ^2 t_{mix}/k)$ lower bound on a quadratic objective driven by a persistent two-state chain. We then extend this framework to heavy-tailed Markovian gradients satisfying a stationary finite-$p$-moment condition, $p \in (1,2]$. We design an all-samples clipped block method that uses every Markov transition while mitigating Markovian bias. Under a transition budget $T$, this algorithm achieves a high-probability stochastic error of $\widetilde{O}(σ_p^2(t_{mix}/T)^{2(p-1)/p})$. We establish a matching lower bound by reducing PL optimization to heavy-tailed mean estimation for a sticky Markov chain. Ultimately, this work tightly characterizes the optimal polynomial dependence on mixing time for light-tailed PL-SGD, and the optimal heavy-tail exponent and effective-sample-size dependence in the robust regime.

Problem

• PL-SGD
• Markovian Noise
• Mixing Time
• High-Probability Bounds
• Heavy-Tailed Gradients

Innovation

• Markovian noise
• Polyak-Łojasiewicz condition
• high-probability convergence
• heavy-tailed gradients
• mixing time