Quasi-optimal hierarchically semi-separable matrix approximation

2025-05-22

AI Summary

This work addresses the problem of efficiently constructing a near-optimal Hierarchically Semi-Separable (HSS) matrix approximation using only matrix-vector multiplications (i.e., black-box queries). We propose the first randomized algorithm that is both theoretically rigorous and practically efficient: it computes a rank-$k$ HSS approximation in polynomial time using only $O(k log(N/k))$ matrix-vector products and $O(Nk^2 log(N/k))$ additional arithmetic operations. The expected Frobenius norm error of the approximation is within an $O(log(N/k))$ factor of the optimal rank-$k$ HSS approximation. This is the first result to establish theoretical guarantees for near-optimal HSS approximation under both explicit-access and black-box query models. Furthermore, we derive explicit error bounds for projection-cost preserving (PCP) sketches, significantly reducing both query complexity and computational overhead for compressing large dense matrices.

Technology Category

• Machine Learning: Matrix & Tensor Methods
• Search and Optimization: Non-convex Optimization
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Security and Privacy: Large-scale security measurements
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines

Abstract

We present a randomized algorithm for producing a quasi-optimal hierarchically semi-separable (HSS) approximation to an $N imes N$ matrix $A$ using only matrix-vector products with $A$ and $A^T$. We prove that, using $O(k log(N/k))$ matrix-vector products and ${O}(N k^2 log(N/k))$ additional runtime, the algorithm returns an HSS matrix $B$ with rank-$k$ blocks whose expected Frobenius norm error $mathbb{E}[|A - B|_F^2]$ is at most $O(log(N/k))$ times worse than the best possible approximation error by an HSS rank-$k$ matrix. In fact, the algorithm we analyze in a simple modification of an empirically effective method proposed by [Levitt&Martinsson, SISC 2024]. As a stepping stone towards our main result, we prove two results that are of independent interest: a similar guarantee for a variant of the algorithm which accesses $A$'s entries directly, and explicit error bounds for near-optimal subspace approximation using projection-cost-preserving sketches. To the best of our knowledge, our analysis constitutes the first polynomial-time quasi-optimality result for HSS matrix approximation, both in the explicit access model and the matrix-vector product query model.

Problem

• Randomized algorithm for quasi-optimal HSS matrix approximation
• Efficient HSS approximation using matrix-vector products
• First polynomial-time quasi-optimality result for HSS approximation

Innovation

• Randomized algorithm for HSS matrix approximation
• Uses matrix-vector products with A and A^T
• Polynomial-time quasi-optimality result