A lower bound for stepsize-based acceleration of gradient descent

2026-08-10

AI Summary

This work investigates whether gradient descent algorithms relying solely on predetermined non-negative stepsize schedules can achieve the optimal $O(T^{-2})$ last-iterate convergence rate in smooth convex optimization. By constructing adversarial instances and employing a refined recursive analysis, the authors establish—for the first time—a lower bound of $\Omega(T^{-1.9319})$ for this class of methods, rigorously demonstrating that stepsize scheduling alone is insufficient to attain the $O(T^{-2})$ rate. This result delineates the fundamental theoretical limitations of stepsize-scheduled gradient descent and fills a critical gap in the lower-bound analysis for such algorithms.

Abstract

Recent work has shown that, for smooth convex optimization, plain gradient descent can be accelerated from its textbook convergence rate of $O(T^{-1})$ (where $T$ denotes the number of iterations) to $O\big(T^{-\log_2(1+\sqrt{2})}\big)$ using carefully designed stepsize schedules alone, without resorting to momentum or other algorithmic modifications. Despite this progress, however, little was known about lower bounds for such methods beyond the classical $Ω(T^{-2})$ benchmark for general first-order methods. In this work, we present a new lower bound of $Ω(T^{-1.9319})$ for the last-iterate convergence rate of gradient descent with predetermined nonnegative stepsize schedules. This result provides rigorous evidence that stepsize schedules alone cannot accelerate plain GD to the optimal $O(T^{-2})$ convergence rate. The proof was developed by GPT-5.6 Sol Pro under the authors' guidance.

Problem

• gradient descent
• stepsize schedule
• convergence rate
• lower bound
• acceleration

Innovation

• stepsize schedule
• gradient descent
• convergence lower bound
• acceleration
• first-order methods