The matrix-vector complexity of $Ax=b$

2026-02-04

AI Summary

This study establishes tight theoretical lower bounds on the number of matrix-vector products required to solve linear systems $Ax = b$. By employing minimax analysis, adversarial constructions, and spectral theory, the authors derive the first complexity lower bounds with explicit constants. Their results show that any algorithm permitted to access both $A$ and its transpose must perform at least $\Omega(\kappa \log(1/\varepsilon))$ operations, where $\kappa$ is the condition number and $\varepsilon$ the desired accuracy. In contrast, algorithms restricted to using only $A$ (one-sided methods) require $n$ iterations even for well-conditioned problems, matching known upper bounds. This work reveals a fundamental distinction between one-sided and two-sided methods in their dependence on accuracy and conditioning, and demonstrates that Krylov subspace methods such as conjugate gradient are optimal under specific settings.

Technology Category

• Machine Learning: Matrix & Tensor Methods
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning
• Search and Optimization: Non-convex Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments

Abstract

Matrix-vector algorithms, particularly Krylov subspace methods, are widely viewed as the most effective algorithms for solving large systems of linear equations. This paper establishes lower bounds on the worst-case number of matrix-vector products needed by such an algorithm to approximately solve a general linear system. The first main result is that, for a matrix-vector algorithm which can perform products with both a matrix and its transpose, $\Omega(\kappa \log(1/\varepsilon))$ matrix-vector products are necessary to solve a linear system with condition number $\kappa$ to accuracy $\varepsilon$, matching an upper bound for conjugate gradient on the normal equations. The second main result is that one-sided algorithms, which lack access to the transpose, must use $n$ matrix-vector products to solve an $n \times n$ linear system, even when the problem is perfectly conditioned. Both main results include explicit constants that match known upper bounds up to a factor of four. These results rigorously demonstrate the limitations of matrix-vector algorithms and confirm the optimality of widely used Krylov subspace algorithms.

Problem

• matrix-vector complexity
• linear systems
• Krylov subspace methods
• lower bounds
• condition number

Innovation

• matrix-vector complexity
• lower bounds
• Krylov subspace methods
• condition number
• linear system solving