Make the most of what you have: Resource-efficient randomized algorithms for matrix computations

2025-12-17

AI Summary

This project addresses efficient randomized algorithms for three fundamental matrix computations under resource constraints: (1) low-rank positive semidefinite (PSD) matrix approximation, aiming for exact low-rank reconstruction with minimal element accesses; (2) estimation of implicit matrix properties—trace, diagonal entries, and row norms—using only matrix-vector products; and (3) numerically robust floating-point solutions to overdetermined linear least-squares problems. Methodologically, we propose a randomized pivoted Cholesky decomposition to drastically reduce element accesses in PSD approximation; introduce a novel leave-one-out framework for property estimation, achieving optimal sample complexity; and design the first randomized least-squares solver that simultaneously attains asymptotic acceleration—O(nd log d) time for n × d matrices—and numerical backward stability, matching the accuracy of direct methods. These contributions advance both theoretical guarantees and practical efficiency in large-scale, memory- or query-constrained numerical linear algebra.

Technology Category

• Machine Learning: Matrix & Tensor Methods
• Reasoning under Uncertainty: Stochastic Optimization
• Search and Optimization: Non-convex Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Security and Privacy: Large-scale security measurements
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

In recent years, randomized algorithms have established themselves as fundamental tools in computational linear algebra, with applications in scientific computing, machine learning, and quantum information science. Many randomized matrix algorithms proceed by first collecting information about a matrix and then processing that data to perform some computational task. This thesis addresses the following question: How can one design algorithms that use this information as efficiently as possible, reliably achieving the greatest possible speed and accuracy for a limited data budget? The first part of this thesis focuses on low-rank approximation for positive-semidefinite matrices. Here, the goal is to compute an accurate approximation to a matrix after accessing as few entries of the matrix as possible. This part of the thesis explores the randomly pivoted Cholesky (RPCholesky) algorithm for this task, which achieves a level of speed and reliability greater than other methods for the same problem. The second part of this thesis considers the task of estimating attributes of an implicit matrix accessible only by matrix-vector products. This thesis describes the leave-one-out approach to developing matrix attribute estimation algorithms and develops optimized trace, diagonal, and row-norm estimation algorithms. The third part of this thesis considers randomized algorithms for overdetermined linear least squares problems. Randomized algorithms for linear-least squares problems are asymptotically faster than any known deterministic algorithm, but recent work has raised questions about the accuracy of these methods in floating point arithmetic. This thesis shows these issues are resolvable by developing fast randomized least-squares problem achieving backward stability, the gold-standard stability guarantee for a numerical algorithm.

Problem

• Designing resource-efficient randomized algorithms for matrix computations.
• Developing accurate low-rank approximations with minimal matrix entry access.
• Ensuring backward stability in randomized linear least squares algorithms.

Innovation

• Randomly pivoted Cholesky algorithm for low-rank approximation
• Leave-one-out approach for matrix attribute estimation
• Fast randomized least-squares with backward stability guarantee