A family of spectral conjugate gradient algorithms derived by least-squares approximations based on a modified quasi--Newton update with application to a revised robust binary classification model

2026-09-11

AI Summary

本文提出了一种基于改进拟牛顿更新的谱共轭梯度算法，用于解决无约束优化问题，并应用于鲁棒二分类模型中以提高准确性和训练效率。

Abstract

We develop a spectral three-term modification of the classic Hestenes--Stiefel conjugate gradient algorithm, preserving its anti-jamming characteristic and, simultaneously, taking care of the sufficient descent property. We discuss how a modified secant equation can be extracted from our modification scheme, yielding a memoryless BFGS updating formula. Then, the spectral parameter of our method is obtained by steering its direction toward the given BFGS direction within a least-squares context. Using our technical improvements, we outline the general framework of our algorithm and discuss its theoretical features, including the descent and convergence properties, without the convexity assumption. We put our algorithm to the test in comparison with the three other conjugate gradient algorithms on a set of CUTEr unconstrained optimization test models, comparing the outputs using the Dolan--More measure. Next, we provide a concise evaluation of the results, highlighting the practical advantages of our algorithm. As a real-world case study, we introduce a reduced quadratic surface SVM with the rescaled loss for robust binary classification and apply the proposed algorithm to assess its accuracy and training time against several other SVMs.

Problem

• conjugate gradient
• unconstrained optimization
• robust binary classification
• BFGS update

Innovation

• spectral conjugate gradient
• modified quasi-Newton update
• least-squares approximations
• robust binary classification
• reduced quadratic surface SVM