Provable Edge-of-Stability for Adam on a One-Dimensional Quadratic

2026-08-20

AI Summary

研究通过分析一维二次函数上Adam优化器的行为，揭示了其趋向稳定阈值的机制及该机制失效的情况，从而解释了边缘稳定性现象。

Abstract

The edge-of-stability (EoS) phenomenon of Adam has been widely observed, while its underlying dynamical mechanism is not yet fully understood. We study uncorrected Adam on a one-dimensional quadratic, a clean setting where constant curvature isolates the optimizer-induced dynamics behind the EoS. We characterize the resulting dynamics across the parameter space. In broad regimes, we prove that Adam exhibits a restoring tendency toward its frozen stability threshold $2(1+β_1)/[η(1-β_1)]$. We also identify settings in which this edge-seeking mechanism breaks down, including strictly subcritical periodic orbits and specially tuned trajectories that converge to the optimum while remaining uniformly supercritical. These results give a concrete dynamical explanation for Adam's EoS in a setting free of evolving loss geometry, while also exposing its limitations.

Problem

• Edge-of-stability
• Adam
• Dynamical Mechanism
• One-dimensional Quadratic
• Optimizer-induced Dynamics

Innovation

• Edge-of-Stability
• Adam Optimizer
• Dynamical Mechanism
• Frozen Stability Threshold
• One-Dimensional Quadratic