Optimality of Approximate Message Passing Algorithms for Spiked Matrix Models with Rotationally Invariant Noise

2024-05-28

arXiv.org

AI Summary

This work addresses the high-dimensional estimation of rank-one signal matrices corrupted by rotationally invariant noise. To overcome the performance limitations of classical PCA and existing iterative methods, we propose a novel class of Approximate Message Passing (AMP) algorithms. For the first time, we establish a rigorous high-dimensional dynamical characterization of these algorithms under rotationally invariant noise. By jointly modeling the spectral structure of the noise and the signal prior, we derive the optimal iterative denoiser and an asymptotically optimal estimator. Theoretically, our algorithm achieves the information-theoretic limit—the minimal asymptotic estimation error—in the spiked matrix model, substantially outperforming PCA and state-of-the-art methods. Moreover, it enjoys provable convergence and asymptotic optimality guarantees.

Technology Category

• Machine Learning: Matrix & Tensor Methods
• Reasoning under Uncertainty: Stochastic Optimization
• Search and Optimization: Non-convex Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Security and Privacy: Large-scale security measurements
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

We study the problem of estimating a rank one signal matrix from an observed matrix generated by corrupting the signal with additive rotationally invariant noise. We develop a new class of approximate message-passing algorithms for this problem and provide a simple and concise characterization of their dynamics in the high-dimensional limit. At each iteration, these algorithms exploit prior knowledge about the noise structure by applying a non-linear matrix denoiser to the eigenvalues of the observed matrix and prior information regarding the signal structure by applying a non-linear iterate denoiser to the previous iterates generated by the algorithm. We exploit our result on the dynamics of these algorithms to derive the optimal choices for the matrix and iterate denoisers. We show that the resulting algorithm achieves the smallest possible asymptotic estimation error among a broad class of iterative algorithms under a fixed iteration budget.

Problem

• Estimating rank-one signal matrices from noisy observations
• Developing optimal approximate message-passing algorithms for spiked models
• Minimizing asymptotic estimation error under fixed iteration constraints

Innovation

• Approximate message-passing algorithms with matrix denoisers
• Exploiting noise structure via eigenvalue nonlinear transformations
• Optimal asymptotic estimation error under iteration constraints