On Universality of Non-Separable Approximate Message Passing Algorithms

2025-06-28

AI Summary

This work addresses the state evolution (SE) universality of non-separable Approximate Message Passing (AMP) algorithms under non-Gaussian, non-i.i.d. measurement matrices. To overcome the limitation of existing SE theory—which requires Gaussian or rotationally invariant inputs—we introduce the *Bounded Composition Property* (BCP) and its approximate variant. This enables, for the first time, SE universality guarantees for AMP with polynomial and Lipschitz-continuous non-separable nonlinearities. By integrating tensor-based modeling, random matrix theory, and the AMP framework, we rigorously prove that nonlinear denoisers satisfying (or asymptotically approximating) BCP preserve SE validity across a broad class of non-Gaussian input matrices. Our result unifies the mean-field behavior of practical algorithms—including local and spectral denoising—and substantially strengthens the theoretical foundation for analyzing complex learning dynamics in modern high-dimensional inference problems.

Technology Category

• Machine Learning: Matrix & Tensor Methods
• Search and Optimization: Non-convex Optimization
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Security and Privacy: Large-scale security measurements
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Mean-field characterizations of first-order iterative algorithms -- including Approximate Message Passing (AMP), stochastic and proximal gradient descent, and Langevin diffusions -- have enabled a precise understanding of learning dynamics in many statistical applications. For algorithms whose non-linearities have a coordinate-separable form, it is known that such characterizations enjoy a degree of universality with respect to the underlying data distribution. However, mean-field characterizations of non-separable algorithm dynamics have largely remained restricted to i.i.d. Gaussian or rotationally-invariant data. In this work, we initiate a study of universality for non-separable AMP algorithms. We identify a general condition for AMP with polynomial non-linearities, in terms of a Bounded Composition Property (BCP) for their representing tensors, to admit a state evolution that holds universally for matrices with non-Gaussian entries. We then formalize a condition of BCP-approximability for Lipschitz AMP algorithms to enjoy a similar universal guarantee. We demonstrate that many common classes of non-separable non-linearities are BCP-approximable, including local denoisers, spectral denoisers for generic signals, and compositions of separable functions with generic linear maps, implying the universality of state evolution for AMP algorithms employing these non-linearities.

Problem

• Study universality of non-separable AMP algorithms
• Identify conditions for universal state evolution
• Demonstrate BCP-approximability for common non-linearities

Innovation

• Non-separable AMP with polynomial non-linearities
• Bounded Composition Property for tensors
• Universal state evolution for non-Gaussian matrices