Orthogonal Approximate Message Passing with Optimal Spectral Initializations for Rectangular Spiked Matrix Models

2025-12-22

AI Summary

We address the signal estimation problem in rectangular spiked matrix models corrupted by general rotationally invariant (RI) noise. We propose the Orthogonal Approximate Message Passing (OAMP) algorithm and establish a rigorous state evolution (SE) theory. Our contributions are threefold: (i) we introduce the first spectral initialization for rectangular settings that jointly exploits multiple spectral outliers; (ii) we construct an iterative, layer-wise optimal denoising framework that asymptotically achieves Bayesian-optimal performance; and (iii) we rigorously prove that the algorithm’s SE trajectory coincides exactly with the replica-symmetric prediction. This establishes statistical optimality of OAMP within the broader class of generalized iterative estimation algorithms. Moreover, the method accommodates non-Gaussian signal priors and arbitrary RI noise distributions, significantly extending the theoretical foundations and applicability of OAMP to non-square matrices and non-Gaussian noise regimes.

Technology Category

• Search and Optimization: Non-convex Optimization
• Machine Learning: Optimization
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Security and Privacy: Large-scale security measurements

Abstract

We propose an orthogonal approximate message passing (OAMP) algorithm for signal estimation in the rectangular spiked matrix model with general rotationally invariant (RI) noise. We establish a rigorous state evolution that precisely characterizes the algorithm's high-dimensional dynamics and enables the construction of iteration-wise optimal denoisers. Within this framework, we accommodate spectral initializations under minimal assumptions on the empirical noise spectrum. In the rectangular setting, where a single rank-one component typically generates multiple informative outliers, we further propose a procedure for combining these outliers under mild non-Gaussian signal assumptions. For general RI noise models, the predicted performance of the proposed optimal OAMP algorithm agrees with replica-symmetric predictions for the associated Bayes-optimal estimator, and we conjecture that it is statistically optimal within a broad class of iterative estimation methods.

Problem

• OAMP algorithm for signal estimation in rectangular spiked matrix models
• Rigorous state evolution to characterize high-dimensional algorithm dynamics
• Optimal spectral initializations under minimal empirical noise spectrum assumptions

Innovation

• OAMP algorithm for rectangular spiked matrix models
• Spectral initializations with minimal empirical noise assumptions
• Combining multiple outliers under non-Gaussian signal conditions