Improved Gradient Descent Lower Bounds Beyond Nesterov

2026-09-02

AI Summary

研究通过预设步长加速梯度下降法在平滑凸优化中的极限，使用改进的理论分析方法证明了更紧的下界。

Abstract

We study how far gradient descent (GD) can be accelerated by predetermined stepsizes in smooth convex optimization. Going beyond the classical $Ω(n^{-2})$ first-order oracle lower bound of Nemirovsky and Yudin, we prove an $Ω(n^{-1.6342})$ non-anytime lower bound and an $Ω(n^{-1.2408})$ anytime lower bound. These improve the recent $Ω(n^{-1.932})$ non-anytime lower bound of Ma and Chen and the $Ω(n^{-4/3})$ anytime lower bound of Tsai et al., respectively. Together with the non-anytime $O(n^{-\log_2(1+\sqrt{2})})$ rate achieved by silver schedules, our anytime lower bound establishes a strict separation between the achievable convergence exponents in the two settings.

Problem

• Gradient Descent
• Smooth Convex Optimization
• Lower Bound

Innovation

• Gradient Descent
• Lower Bounds
• Smooth Convex Optimization
• Pre-determined Stepsizes