Accelerating Power Method with Fast Sketching for Stronger Low-Rank Approximation

2026-05-10

AI Summary

This work addresses the computational bottleneck of power iteration in low-rank approximation under high target ranks, where expensive matrix multiplications dominate the cost. To overcome this limitation, the authors propose an efficient algorithmic framework that integrates fast sketching techniques with power iteration, substantially reducing computational overhead while remaining applicable to singular value decomposition, low-rank approximation, and Nyström methods. The key theoretical contribution lies in a novel analysis based on regularized spectral approximation, which offers greater flexibility than conventional arguments and enables rigorous generalization of theoretical guarantees for power iteration. Empirical evaluations on standard benchmark datasets demonstrate that the proposed method achieves excellent performance, algorithmic simplicity, and provable efficiency.

Abstract

The power method is one of the most fundamental tools for extracting top principal components from data through low-rank matrix approximation. Yet, when the target rank is large, the cost of matrix multiplication associated with this procedure becomes a major bottleneck. We develop an algorithmic and theoretical framework for accelerating the power method using fast sketching, which is a popular paradigm in randomized linear algebra. Our framework leads to simple and provably efficient methods for singular value decomposition, low-rank factorization, and Nyström approximation, which attain strong numerical performance on benchmark problems. The key novelty in our analysis is the use of regularized spectral approximation, a property of fast sketching methods which proves more flexible in generalizing power method guarantees than traditional arguments.

Problem

• power method
• low-rank approximation
• matrix multiplication
• large target rank
• computational bottleneck

Innovation

• fast sketching
• power method
• regularized spectral approximation
• low-rank approximation
• randomized linear algebra