Improved Sparse Recovery for Approximate Matrix Multiplication

2026-02-04

AI Summary

This work addresses the problem of efficient approximate matrix multiplication with reduced computational complexity while controlling error relative to the Frobenius norm of the output matrix. The authors propose a novel algorithm based on pseudorandom rotations, integrating the fast Hadamard transform with asymmetric diagonal scaling to uniformly redistribute the output norm across matrix entries, complemented by a randomized sparse recovery mechanism. Under an unbiased estimation guarantee, the method achieves a runtime of $O(n^2(r + \log n))$ and incurs a total mean squared error of $(1 - r/n)\|AB\|_F^2$ in the biased setting or $(n/r)\|AB\|_F^2$ in the unbiased case. Compared to existing approaches, it offers comparable or improved accuracy with a speedup by a logarithmic factor.

Technology Category

• Machine Learning: Matrix & Tensor Methods
• Search and Optimization: Non-convex Optimization
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Security and Privacy: Large-scale security measurements
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines

Abstract

We present a simple randomized algorithm for approximate matrix multiplication (AMM) whose error scales with the *output* norm $\|AB\|_F$. Given any $n\times n$ matrices $A,B$ and a runtime parameter $r\leq n$, the algorithm produces in $O(n^2(r+\log n))$ time, a matrix $C$ with total squared error $\mathbb{E}[\|C-AB\|_F^2]\le (1-\frac{r}{n})\|AB\|_F^2$, per-entry variance $\|AB\|_F^2/n^2$ and bias $\mathbb{E}[C]=\frac{r}{n}AB$. Alternatively, the algorithm can compute an *unbiased* estimation with expected total squared error $\frac{n}{r}\|{AB}\|_{F}^2$, recovering the state-of-art AMM error obtained by Pagh's TensorSketch algorithm (Pagh, 2013). Our algorithm is a log-factor faster. The key insight in the algorithm is a new variation of pseudo-random rotation of the input matrices (a Fast Hadamard Transform with asymmetric diagonal scaling), which redistributes the Frobenius norm of the *output* $AB$ uniformly across its entries.

Problem

• approximate matrix multiplication
• sparse recovery
• Frobenius norm
• randomized algorithm
• error scaling

Innovation

• approximate matrix multiplication
• sparse recovery
• pseudo-random rotation
• Fast Hadamard Transform
• Frobenius norm redistribution