Global Convergence of Adaptive Sensing for Principal Eigenvector Estimation

2025-05-16

AI Summary

Existing subspace tracking algorithms for principal eigenvector estimation in high-dimensional data streams rely on full-dimensional observations, rendering them unsuitable for compressed sensing scenarios. Method: We propose the Adaptive Sensing Oja algorithm, which requires only two compressive measurements per iteration—along the current estimate direction and a randomly sampled orthogonal direction—and achieves global convergence under noise. Contributions: (i) First rigorous global convergence guarantee for noisy subspace tracking under adaptive compression; (ii) Identification of a two-phase convergence behavior—initial warm-up followed by asymptotic decay; (iii) Tight analysis showing the sine error decays at rate $O(lambda_1lambda_2 d^2/(Delta^2 t))$, matching the minimax lower bound up to a factor of $d$; (iv) A significantly simplified analytical framework. The method enables real-time, high-dimensional streaming data analysis where full-dimensional sampling is infeasible.

Technology Category

• Machine Learning: Learning with Manifolds
• Search and Optimization: Non-convex Optimization
• Data Mining & Knowledge Management: Data Compression

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Security and Privacy: Large-scale security measurements
• Systems and Infrastructure for Web, Mobile and WoT: Data management and stream processing for Web, mobile and wireless applications

Abstract

This paper addresses the challenge of efficient principal component analysis (PCA) in high-dimensional spaces by analyzing a compressively sampled variant of Oja's algorithm with adaptive sensing. Traditional PCA methods incur substantial computational costs that scale poorly with data dimensionality, whereas subspace tracking algorithms like Oja's offer more efficient alternatives but typically require full-dimensional observations. We analyze a variant where, at each iteration, only two compressed measurements are taken: one in the direction of the current estimate and one in a random orthogonal direction. We prove that this adaptive sensing approach achieves global convergence in the presence of noise when tracking the leading eigenvector of a datastream with eigengap $Delta=lambda_1-lambda_2$. Our theoretical analysis demonstrates that the algorithm experiences two phases: (1) a warmup phase requiring $O(lambda_1lambda_2d^2/Delta^2)$ iterations to achieve a constant-level alignment with the true eigenvector, followed by (2) a local convergence phase where the sine alignment error decays at a rate of $O(lambda_1lambda_2d^2/Delta^2 t)$ for iterations $t$. The guarantee aligns with existing minimax lower bounds with an added factor of $d$ due to the compressive sampling. This work provides the first convergence guarantees in adaptive sensing for subspace tracking with noise. Our proof technique is also considerably simpler than those in prior works. The results have important implications for applications where acquiring full-dimensional samples is challenging or costly.

Problem

• Efficient PCA in high-dimensional spaces using adaptive sensing
• Global convergence of compressed Oja's algorithm with noise
• Reducing computational costs in subspace tracking with compressive measurements

Innovation

• Adaptive sensing with compressed measurements
• Global convergence in noisy environments
• Two-phase convergence analysis