Numerical Linear Algebra in Linear Space

2025-07-03

AI Summary

This paper addresses the efficient solution of integer linear systems $Ax = b$ over finite-dimensional linear spaces. We present the first randomized solver for polynomially bounded integer inputs that is independent of the matrix condition number. Our algorithm operates over the rationals, achieving $ ilde{O}(n^2 cdot ext{nnz}(A))$ time complexity and $O(n log n)$ workspace—breaking the classical accuracy–space trade-off. The core techniques integrate bit-complexity optimization, near-linear-space data structures, and a synergistic design of exact integer arithmetic with approximation theory. As a result, our method provides a unified, highly efficient primitive for fundamental numerical tasks—including linear regression, linear programming, and eigenvalue/singular value decomposition—delivering exact or high-accuracy approximate solutions in polynomial time while using near-linear space. This significantly expands the tractability frontier for large-scale numerical linear algebra problems.

Technology Category

• Machine Learning: Matrix & Tensor Methods
• Reasoning under Uncertainty: Stochastic Optimization
• Constraint Satisfaction and Optimization: Solvers and Tools

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines

Abstract

We present a randomized linear-space solver for general linear systems $mathbf{A} mathbf{x} = mathbf{b}$ with $mathbf{A} in mathbb{Z}^{n imes n}$ and $mathbf{b} in mathbb{Z}^n$, without any assumption on the condition number of $mathbf{A}$. For matrices whose entries are bounded by $mathrm{poly}(n)$, the solver returns a $(1+ε)$-multiplicative entry-wise approximation to vector $mathbf{x} in mathbb{Q}^{n}$ using $widetilde{O}(n^2 cdot mathrm{nnz}(mathbf{A}))$ bit operations and $O(n log n)$ bits of working space (i.e., linear in the size of a vector), where $mathrm{nnz}$ denotes the number of nonzero entries. Our solver works for right-hand vector $mathbf{b}$ with entries up to $n^{O(n)}$. To our knowledge, this is the first linear-space linear system solver over the rationals that runs in $widetilde{O}(n^2 cdot mathrm{nnz}(mathbf{A}))$ time. We also present several applications of our solver to numerical linear algebra problems, for which we provide algorithms with efficient polynomial running time and near-linear space. In particular, we present results for linear regression, linear programming, eigenvalues and eigenvectors, and singular value decomposition.

Problem

• Solves general linear systems with integer matrices efficiently
• Achieves linear-space complexity for rational solution vectors
• Applies to numerical problems like regression and SVD

Innovation

• Randomized linear-space solver for general systems
• Handles matrices with poly(n)-bounded entries
• Efficient polynomial time with near-linear space