Well-Conditioned Oblivious Perturbations in Linear Space

2026-04-25

AI Summary

This work proposes a novel sparse perturbation method that significantly reduces the computational and storage costs associated with traditional Gaussian perturbations, which require O(n²) random variables to improve matrix condition numbers. By introducing a structured pattern matrix and a non-uniformly dependent sparse random perturbation, the proposed approach achieves a condition number of O(n) for any deterministic matrix using only O(n) random numbers and O(log n) bits of precision. Key contributions include the construction of the pattern matrix, a new lower bound on the smallest singular value for dependent random matrices, and an efficient smoothed analysis framework. The method matches the condition-number improvement of Gaussian perturbations while drastically lowering perturbation generation cost, enabling conjugate gradient methods for linear systems to converge in just O(n) matrix-vector multiplications.

Technology Category

• Machine Learning: Matrix & Tensor Methods
• Reasoning under Uncertainty: Stochastic Optimization
• Search and Optimization: Non-convex Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Security and Privacy: Large-scale security measurements

Abstract

Perturbing a deterministic $n$-dimensional matrix with small Gaussian noise is a cornerstone of smoothed analysis of algorithms [Spielman and Teng, JACM 2004], as it reduces the condition number of the input to $O(n)$, and with it the complexity of many matrix algorithms. However, when deployed algorithmically, these perturbations are expensive due to the cost of generating and storing $n^2$ Gaussian random variables. We propose a perturbation that requires generating and storing $O(n)$ random numbers in $O(\log n)$ bits of precision, and reduces the condition number of any deterministic matrix to $O(n)$, matching Gaussian perturbations. Our result in particular implies a better complexity for the perturbed conjugate gradient algorithm, showing that we can solve an $n\times n$ linear system in linear space to within an arbitrarily small constant backward error using $O(n)$ matrix-vector products. In our construction, we introduce the concept of a pattern matrix, which is a dense deterministic matrix that maps all sparse vectors into dense vectors, and we combine it with a sparse perturbation whose entries are dependent and located in a non-uniform fashion. In order to analyze this construction, we develop new techniques for lower bounding the smallest singular value of a random matrix with dependent entries.

Problem

• condition number
• matrix perturbation
• smoothed analysis
• linear space
• randomized algorithms

Innovation

• oblivious perturbation
• condition number
• pattern matrix
• dependent random entries
• smoothed analysis