Non-Expansive Two-Time-Scale Stochastic Approximation: A Fixed-Schedule One-Quarter Barrier and Bias-Corrected Acceleration

2026-07-14

AI Summary

This work addresses the convergence rate bottleneck in two-timescale stochastic approximation caused by nonexpansive slow mappings. Focusing on settings where the fast mapping is contractive while the slow mapping is merely nonexpansive, the authors establish a tight lower bound on the Krasnosel’skiĭ–Mann residual under constant step sizes, revealing a fundamental $k^{-1/4}$ rate barrier. To overcome this limitation, they propose a bias-corrected preconditioning mechanism that reduces the first-order bias of the slow mapping to second order and introduce a single-loop algorithm that avoids the computational overhead of nested inner iterations. Theoretical analysis shows that the nested algorithm with bias correction improves sample complexity from $T^{-1/4 + o(1)}$ to $T^{-1/3 + o(1)}$, while the single-loop variant achieves the optimal $T^{-1/2 + o(1)}$ convergence rate with only $O(1)$ primal samples per iteration.

Abstract

Non-expansive two-time-scale stochastic approximation is governed by a slow stochastic Krasnoselskii--Mann fixed-point iteration rather than by contraction to a unique equilibrium. We study this regime under a contractive fast map and a non-expansive reduced slow map. We first prove a finite-horizon lower bound showing that, for any prescribed slow stepsize schedule $(β_k)$, the classical KM residual scale $(\sum_{i<N}β_i(1-β_i))^{-1}$ is worst-case sharp for the corresponding unregularized KM update. Combined with the raw fast-tracking leakage scale, this explains the previously observed $k^{-1/4+o(1)}$ last-iterate mean-square residual exponent. We then introduce a residual-preconditioned slow oracle that cancels the first-order dependence on the fast tracking error. In a nested Tikhonov-KM algorithm, the uncorrected oracle yields total-sample rate $T^{-1/4+o(1)}$, while the corrected oracle yields $T^{-1/3+o(1)}$. This improvement comes from changing the slow-oracle bias from first order to second order in the fast error after all inner-loop samples are counted. Finally, we show that the repeated inner-loop cost of the nested method can be avoided in a smooth derivative-oracle model. A single-loop algorithm that tracks both the fast equilibrium and the leakage preconditioner online achieves $T^{-1/2+o(1)}$ with $O(1)$ primitive samples per iteration.

Problem

• two-time-scale stochastic approximation
• non-expansive mapping
• Krasnoselskii–Mann iteration
• convergence rate
• bias correction

Innovation

• two-time-scale stochastic approximation
• Krasnoselskii–Mann iteration
• bias-corrected acceleration
• residual preconditioning
• single-loop algorithm