Silver Rate Is (Almost) Optimal for Gradient Descent Acceleration

2026-09-08

AI Summary

研究通过预设非负步长加速梯度下降法在光滑凸优化中的效果，证明了银率几乎是最佳的收敛速度。

Abstract

We study how far gradient descent (GD) can be accelerated by predetermined nonnegative stepsizes in smooth convex optimization. Writing $p_{\mathrm{sil}}=\log_2(1+\sqrt{2})$, we prove an $\Omega\left(n^{-p_{\mathrm{sil}}-O(\sqrt{\log\log n/\log n})}\right)$ non-anytime lower bound. In the anytime setting, every infinite nonnegative schedule has infinitely many horizons with error $\Omega\left(n^{-\frac{2p_{\mathrm{sil}}}{1+p_{\mathrm{sil}}}-O(\sqrt{\log\log n/\log n})}\right)$. Together with the silver-schedule upper bound [Altschuler and Parrilo, 2025] and the anytime upper bound [Zhang et al., 2025], our results determine the optimal polynomial convergence exponents in both settings.

Problem

• Gradient Descent
• Smooth Convex Optimization
• Acceleration
• Predetermined Stepsizes
• Convergence Exponents

Innovation

• Gradient Descent
• Acceleration
• Silver Rate
• Optimal Convergence
• Smooth Convex Optimization