Primal Acceleration of Newton's Method

2026-08-21

AI Summary

本文提出了一种新的直接加速牛顿法，用于最小化Hessian Lipschitz连续的凸函数，仅使用原始变量和每次迭代一次线性求解，达到O(1/k^3)的全局收敛率。

Abstract

We develop a new direct accelerated Newton method for minimizing convex functions with Lipschitz continuous Hessian. The algorithm uses only primal variables and performs just one linear solve per iteration. With a simple predetermined choice of parameters, it achieves the global convergence rate of $O(1/k^3)$ in terms of the functional residual. To the best of our knowledge, this is the first second-order method for this problem class attaining this rate while relying solely on one linear system solve per iteration (without solving auxiliary nonlinear regularized subproblems, such as cubic regularization, performing nonlinear parameter searches, or using dual extragradient corrections). Our method can be implemented in a Hessian-free way, using an inexact linear system solver, while preserving the fast global rate. We further extend our construction to arbitrary geometry through Bregman divergence, and to composite optimization problems.

Problem

• Newton's Method
• Convex Functions
• Lipschitz Continuous Hessian
• Primal Variables
• Global Convergence Rate

Innovation

• accelerated Newton method
• primal variables
• Lipschitz continuous Hessian
• global convergence rate
• Hessian-free implementation