Optimal Preconditioning is a Geodesically Convex Optimization Problem

2025-12-06

AI Summary

This work addresses the construction of approximately optimal preconditioners for linear and nonlinear systems to minimize condition numbers. We propose the first unified framework that models condition-number minimization under structured transformations as a geodesically convex optimization problem over unitarily invariant norms. We establish, for the first time, the geodesic convexity of the local condition number of polynomial systems under symmetric Lie group actions, enabling the design of the first preconditioning algorithm with provable convergence guarantees. Leveraging Riemannian gradient updates and Lie group actions, we develop efficient first-order algorithms for Frobenius condition-number optimization, achieving convergence rates of $widetilde{O}(1/varepsilon^2)$ and $widetilde{O}(kappa_F^2 log(1/varepsilon))$, respectively. Both theoretical analysis and empirical evaluation validate the efficacy of our approach.

Technology Category

• Search and Optimization: Non-convex Optimization
• Machine Learning: Optimization
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

We introduce a unified framework for computing approximately-optimal preconditioners for solving linear and non-linear systems of equations. We demonstrate that the condition number minimization problem, under structured transformations such as diagonal and block-diagonal preconditioners, is geodesically convex with respect to unitarily invariant norms, including the Frobenius and Bombieri--Weyl norms. This allows us to introduce efficient first-order algorithms with precise convergence guarantees. For linear systems, we analyze the action of symmetric Lie subgroups $G subseteq GL_m(CC) imes GL_n(CC)$ on the input matrix and prove that the logarithm of the condition number is a smooth geodesically convex function on the associated Riemannian quotient manifold. We obtain explicit gradient formulas, show Lipschitz continuity, and prove convergence rates for computing the optimal Frobenius condition number: $widetilde{O}(1/eps^2)$ iterations for general two-sided preconditioners and $widetilde{O}(κ_F^2 log(1/eps))$ for strongly convex cases such as left preconditioning. We extend our framework to consider preconditioning of polynomial systems $f(x) = 0$, where $f$ is a system of multivariate polynomials. We analyze the local condition number $μ(f, ξ)$, at a root $ξ$ and prove that it also admits a geodesically convex formulation under appropriate group actions. We deduce explicit formulas for the Riemannian gradients and present convergence bounds for the corresponding optimization algorithms. To the best of our knowledge, this is the first preconditioning algorithm with theoretical guarantees for polynomial systems.

Problem

• Optimizes preconditioners for linear and nonlinear equation systems.
• Minimizes condition number using geodesically convex optimization methods.
• Extends framework to polynomial systems with theoretical guarantees.

Innovation

• Geodesic convexity for condition number minimization
• First-order algorithms with explicit convergence guarantees
• Extension to polynomial systems with Riemannian gradients