Sharp Analysis of Power Iteration for Tensor PCA

2024-01-02

Journal of machine learning research

AI Summary

This work investigates the finite-step convergence dynamics of randomly initialized power iteration in tensor PCA. Addressing the planted signal recovery problem, it establishes the first tight upper and lower bounds on convergence within polynomial iterations. Theoretically, it reveals that the practical algorithmic threshold is lower than the previously conjectured one by a $mathrm{polylog}(n)$ factor—correcting a long-standing misconception. It further proposes an adaptive stopping criterion with provably high correlation to the true signal. Methodologically, the analysis integrates high-dimensional probability, spectral theory, and iterative dynamical modeling to rigorously characterize the phase transition in convergence across a broad signal-to-noise ratio regime. Extensive experiments validate both the sharpness of the theoretical bounds and the practical efficacy of the stopping rule. This work delivers the first practically deployable, iteratively computed tensor PCA algorithm with rigorous finite-sample guarantees.

Technology Category

• Machine Learning: Matrix & Tensor Methods
• Search and Optimization: Non-convex Optimization
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Security and Privacy: Large-scale security measurements
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

We investigate the power iteration algorithm for the tensor PCA model introduced in Richard and Montanari (2014). Previous work studying the properties of tensor power iteration is either limited to a constant number of iterations, or requires a non-trivial data-independent initialization. In this paper, we move beyond these limitations and analyze the dynamics of randomly initialized tensor power iteration up to polynomially many steps. Our contributions are threefold: First, we establish sharp bounds on the number of iterations required for power method to converge to the planted signal, for a broad range of the signal-to-noise ratios. Second, our analysis reveals that the actual algorithmic threshold for power iteration is smaller than the one conjectured in literature by a polylog(n) factor, where n is the ambient dimension. Finally, we propose a simple and effective stopping criterion for power iteration, which provably outputs a solution that is highly correlated with the true signal. Extensive numerical experiments verify our theoretical results.

Problem

• Analyzing power iteration convergence for tensor PCA
• Determining algorithmic threshold for signal recovery
• Proposing stopping criterion for power iteration

Innovation

• Analyzing randomly initialized tensor power iteration dynamics
• Establishing sharp convergence bounds for various SNR ranges
• Proposing effective stopping criterion for signal correlation