🤖 AI Summary
For large-scale optimization problems lacking second-order derivative information—such as large-scale data fitting and curve modeling—this paper proposes a parameterized compact Hessian approximation method that replaces the dense Hessian with a low-rank structure, significantly improving scalability. The method introduces the first unified parameterized framework that encompasses and generalizes classical compact quasi-Newton formulas, including BFGS and SR1, and incorporates a flexible vector selection mechanism to better adapt to heterogeneous problem structures. It integrates limited-memory techniques, low-rank matrix decomposition, and quasi-Newton update theory, with task-specific designs for eigenvalue computation, tensor decomposition, and nonlinear regression. Experimental results demonstrate that the proposed method maintains convergence accuracy while substantially reducing memory consumption and computational complexity compared to standard dense or conventional compact Hessian approximations.
📝 Abstract
For minimization problems without 2nd derivative information, methods that estimate Hessian matrices can be very effective. However, conventional techniques generate dense matrices that are prohibitive for large problems. Limited-memory compact representations express the dense arrays in terms of a low rank representation and have become the state-of-the-art for software implementations on large deterministic problems. We develop new compact representations that are parameterized by a choice of vectors and that reduce to existing well known formulas for special choices. We demonstrate effectiveness of the compact representations for large eigenvalue computations, tensor factorizations and nonlinear regressions.