A Lower Bound for the Heavy-Ball Method on Smooth Convex Functions

2026-09-08

AI Summary

本文证明了经典Heavy-Ball方法即使采用非平稳、依赖于时间范围的参数调整，也无法在所有光滑凸函数上达到Nesterov的O(T^-2)收敛速率。

Abstract

Can the classical Heavy-Ball method, with arbitrary horizon-dependent parameters chosen in advance, achieve Nesterov's $O(T^{-2})$ last-iterate rate on every smooth convex objective? We provide a negative answer. For every horizon $T\ge2$ and every predetermined schedule with nonnegative step sizes and momenta in $[0,1)$, there exists a convex $1$-smooth objective, with initialization distance at most one and zero initial velocity, for which the last iterate of the Heavy-Ball method satisfies \[ f(x_T)-f^\star=\Omega\!\left(\frac{1}{T^\alpha\log T}\right), \qquad \alpha=\frac{1+\sqrt5}{2}. \] Thus even fully nonstationary, horizon-dependent tuning cannot give the classical Heavy-Ball method a Nesterov-rate guarantee on the smooth convex class.

Problem

• Heavy-Ball method
• smooth convex functions
• Nesterov's rate
• last-iterate rate
• horizon-dependent parameters

Innovation

• Heavy-Ball method
• last-iterate rate
• smooth convex functions
• Nesterov's rate
• nonstationary tuning