Oblivious Subspace Injection Is Not Enough for Relative Error

2026-04-11

AI Summary

This work investigates whether randomized sketching methods satisfying only Oblivious Subspace Embedding (OSI) can guarantee relative error bounds. By constructing counterexamples in the Frobenius norm for both low-rank approximation and least squares problems, it demonstrates for the first time that OSI alone is insufficient to yield relative error guarantees analogous to those provided by Oblivious Subspace Embeddings (OSEs). The key deficiency lies in the lack of upper control over the optimal residual or tail components. To address this, the study introduces additional conditions that restore near-relative-error performance and naturally extends OSI to the ℓₚ norm setting. Combining tools from random matrix theory, numerical linear algebra, and adversarial construction, this research deepens the understanding of error mechanisms in randomized SVD within the sketch-and-solve framework.

Technology Category

• Machine Learning: Other Foundations of Machine Learning
• Search and Optimization: Non-convex Optimization
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Oblivious subspace injection (OSI) was introduced by Camaño, Epperly, Meyer, and Tropp in 2025 as a much weaker sketching property than oblivious subspace embedding (OSE) that still yields constant-factor guarantees for randomized low-rank approximation and sketch-and-solve least-squares regression. At the Simons Institute in Berkeley during a workshop in October 2025, it was asked whether OSIs also imply relative error bounds rather than just constant-factor guarantees. We show that, from a theoretical standpoint, OSI alone does not yield OSE-style relative-error guarantees whose failure probability is controlled solely by the OSI failure parameter, even though OSI sketches often perform extremely well in practice. We provide counterexamples showing this for sketch-and-solve least squares and for randomized SVD in the Frobenius norm. The missing ingredient from a sketch satisfying only OSI is upper control on the optimal residual or tail component, and when one ensures the sketch has this additional property, a near-relative-error bound is recovered. We also show that there is a natural $\ell_p$ analogue of OSI giving constant-factor sketch-and-solve bounds.

Problem

• Oblivious Subspace Injection
• relative error
• sketching
• low-rank approximation
• least-squares regression

Innovation

• Oblivious Subspace Injection
• relative error bounds
• sketch-and-solve
• randomized SVD
• subspace embedding