A Tight Lower Bound for Smooth Nonconvex Stochastic Optimization with Bounded Gradient Noise

2026-08-09

AI Summary

This work investigates the optimal oracle complexity of smooth nonconvex stochastic optimization under the assumption that gradient noise is uniformly bounded. By constructing an adversarial distribution and leveraging information-theoretic lower-bound techniques, the authors establish—for the first time—that any stochastic adaptive algorithm in the single-sample oracle model requires at least Ω(ΔL/ε² + ΔLσ²/ε⁴) queries to guarantee an expected gradient norm no greater than ε. This lower bound exactly matches the best-known upper bounds, thereby confirming the optimality of existing algorithms. Moreover, the result resolves the open question of whether almost surely bounded oracle error can lead to improved convergence rates, demonstrating that it does not yield any asymptotic advantage over standard stochastic settings.

Abstract

We prove a sharp lower bound for smooth nonconvex stochastic optimization with uniformly bounded gradient noise. In the \(K=1\) fresh-sample model, every randomized adaptive algorithm requires $$Ω\left( \frac{ΔL}{ε^2} + \frac{ΔLσ^2}{ε^4} \right)$$ queries to find a point with expected gradient norm at most \(ε\). This matches the standard upper bound and, to the best of our knowledge, resolves the question raised by [Arjevani et al. 2023] of whether almost-surely bounded oracle error permits a better rate than bounded variance. The proof was independently generated with GPT-5.6 Sol in Codex's Ultra mode during a two-hour session. The human author supplied the prompt and was responsible only forchecking the proof and revising and polishing the manuscript.

Problem

• smooth nonconvex optimization
• stochastic optimization
• gradient noise
• lower bound
• query complexity

Innovation

• nonconvex optimization
• stochastic optimization
• lower bound
• gradient noise
• complexity