Quasi-Quadratic Gradient: A New Direction for Accelerating the BFGS Method in Quasi-Newton Optimization

2026-04-26

AI Summary

This work addresses the limited convergence rate of the BFGS algorithm in quasi-Newton methods by proposing a novel search direction termed the quasi-quadratic gradient (QQG). The QQG method explicitly incorporates local second-order curvature information into the construction of the search direction for the first time, achieved by dynamically correcting the optimization trajectory through multiplication of the inverse Hessian approximation with the current gradient. While preserving the computational efficiency inherent to BFGS, this approach significantly accelerates convergence. Numerical experiments demonstrate that QQG achieves faster convergence rates than standard BFGS across a variety of benchmark problems, thereby validating the effectiveness and superiority of explicitly leveraging local curvature information in the design of search directions.

Technology Category

• Search and Optimization: Non-convex Optimization
• Machine Learning: Optimization
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

In this paper, we introduce the Quasi-Quadratic Gradient (QQG), a novel search direction designed to accelerate the BFGS method within the quasi-Newton framework. By defining the QQG as the product of the inverse Hessian approximation and the current gradient, we explicitly leverage local second-order curvature to rectify the search path. Theoretical analysis and empirical results demonstrate that our approach significantly outperforms vanilla BFGS in convergence speed while maintaining computational efficiency.

Problem

• BFGS
• quasi-Newton
• optimization
• convergence acceleration

Innovation

• Quasi-Quadratic Gradient
• BFGS
• quasi-Newton
• search direction
• curvature information