Sharp analysis of sketched least squares and randomized low-rank approximation

2026-05-18

AI Summary

This study investigates the optimality of random embeddings in sketch-and-solve least squares and randomized SVD for low-rank approximation. By integrating tools from random matrix theory, minimax analysis, and rotational invariance principles, the authors establish that random orthogonal matrices achieve minimax optimality within the sketch-and-solve framework, while any rotationally invariant embedding is optimal for randomized SVD. Building on these theoretical insights, they derive the tightest error bounds to date and corroborate their findings through numerical experiments, demonstrating that a variety of commonly used embeddings closely approach the theoretically optimal performance in practice—thereby revealing a striking universality phenomenon across different embedding schemes.

Abstract

Two widely used randomized algorithms are the sketch-and-solve method for least-squares regression and the randomized SVD for low-rank approximation. These algorithms apply a random embedding to compress a target matrix, and they perform computations on the compressed matrix to save computational cost. This paper asks, what is the optimal random embedding in these algorithms? Also, what is the sharpest possible error bound for the optimal embedding? The paper proves that a random orthonormal matrix is minimax optimal for the sketch-and-solve algorithm while any rotation-invariant embedding is minimax optimal for the randomized SVD. Following these results, the paper obtains the best possible error bounds for sketched least-squares and the randomized SVD. Last, empirical experiments provide evidence of universality phenomena, in which several random embeddings lead to similar accuracy to the optimal embeddings in practice.

Problem

• randomized algorithms
• sketch-and-solve
• randomized SVD
• random embedding
• error bound

Innovation

• randomized sketching
• minimax optimality
• rotation-invariant embedding
• sharp error bounds
• randomized SVD