🤖 AI Summary
This work addresses strongly convex–strongly concave (SCSC) saddle-point optimization. We propose Multi-Greedy SR1 for Saddle-Point problems (MGSR1-SP), the first method to integrate a multi-greedy subspace selection strategy into the symmetric rank-one (SR1) quasi-Newton framework. By adaptively selecting subspaces, MGSR1-SP achieves high-fidelity approximation of the asymmetric, indefinite Hessian square matrix—significantly improving Hessian approximation quality and algorithmic stability. Theoretically, MGSR1-SP attains both global linear and local quadratic convergence rates, with stronger convergence guarantees than existing quasi-Newton and first-order saddle-point methods. Empirically, on canonical tasks including AUC maximization and adversarial debiasing, MGSR1-SP consistently outperforms state-of-the-art approaches in both convergence speed and solution accuracy. These results demonstrate its effectiveness and superiority for machine learning problems requiring efficient second-order information modeling.
📝 Abstract
This paper introduces the Multiple Greedy Quasi-Newton (MGSR1-SP) method, a novel approach to solving strongly-convex-strongly-concave (SCSC) saddle point problems. Our method enhances the approximation of the squared indefinite Hessian matrix inherent in these problems, significantly improving both stability and efficiency through iterative greedy updates. We provide a thorough theoretical analysis of MGSR1-SP, demonstrating its linear-quadratic convergence rate. Numerical experiments conducted on AUC maximization and adversarial debiasing problems, compared with state-of-the-art algorithms, underscore our method's enhanced convergence rate. These results affirm the potential of MGSR1-SP to improve performance across a broad spectrum of machine learning applications where efficient and accurate Hessian approximations are crucial.