A multilevel sketch-and-solve method for overdetermined least squares problems

2026-05-25

AI Summary

This work proposes the first multilevel Sketch-and-Solve framework for severely over-determined large-scale linear least squares problems. The method systematically enhances approximation accuracy by iteratively combining coarse solutions from small sketches with correction terms derived from larger sketches. Theoretical analysis demonstrates that the variance of these correction terms decays rapidly across levels, providing a rigorous foundation for the observed accuracy gains. By integrating randomized sketching, multilevel Monte Carlo estimation, and a careful variance–complexity trade-off, the proposed approach substantially reduces estimator variance. Although its computational cost is modestly higher than that of simple averaging strategies, the significant improvement in solution accuracy establishes a strong basis for future efficient implementations.

Abstract

Sketch-and-solve (SAS) is a very successful method to efficiently estimate the solution of heavily overdetermined large linear least squares problems. It uses random sketching to reduce the size of the problem, hence reducing the computational cost. Several authors have shown that averaging several solutions from SAS further improves the accuracy, which is measured by the residual associated to the approximate solution. Going further, we combine solutions from sketch-and-solve in a multilevel manner, such that the approximate solution is a combination of SAS samples obtained from small sketches and more accurate correction terms obtained from larger sketches. We first consider the variance of the estimator, which depends on the variance of the coarse samples and the correction terms. We show that the variance of the correction terms on each level follows a trend and decreases faster than the variance of the simple SAS estimator. However, we then show that the overall computational cost of our multilevel framework is slightly higher than that of the simple average estimator, so a naive application of multilevel methods appears unattractive for least squares problems.

Problem

• overdetermined least squares
• sketch-and-solve
• multilevel methods
• random sketching
• computational efficiency

Innovation

• multilevel sketch-and-solve
• overdetermined least squares
• random sketching
• variance reduction
• hierarchical correction