Markov Chains Approximate Message Passing

2025-12-02

AI Summary

This paper studies Bayesian inference in the spiked Wigner model—recovering a planted Boolean spike from a noisy symmetric matrix. To bridge the unclear connection between Markov chain Monte Carlo (MCMC) and approximate message passing (AMP), we propose the Restricted Gaussian Dynamics (RGD) auxiliary chain, which rigorously establishes, for the first time, a one-dimensional recursive equivalence between Glauber dynamics on the annealed posterior and AMP iterations. We prove that RGD converges rapidly from a warm start to the correlation-space fixed point, achieving Bayes-optimal recovery. Moreover, under the Sherrington–Kirkpatrick (SK) model’s mixture assumption, RGD reproduces the nontrivial inference phase transition threshold. This work provides the first dynamical-level unifying framework linking MCMC and variational inference, offering rigorous insights into the interplay between sampling-based and iterative deterministic algorithms in high-dimensional Bayesian estimation.

Technology Category

• Machine Learning: Probabilistic Circuits and Graphical Models
• Reasoning under Uncertainty: Relational Probabilistic Models
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Web Mining and Content Analysis: Models for Web evolution
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems

Abstract

Markov chain Monte Carlo algorithms have long been observed to obtain near-optimal performance in various Bayesian inference settings. However, developing a supporting theory that make these studies rigorous has proved challenging. In this paper, we study the classical spiked Wigner inference problem, where one aims to recover a planted Boolean spike from a noisy matrix measurement. We relate the recovery performance of Glauber dynamics on the annealed posterior to the performance of Approximate Message Passing (AMP), which is known to achieve Bayes-optimal performance. Our main results rely on the analysis of an auxiliary Markov chain called restricted Gaussian dynamics (RGD). Concretely, we establish the following results: 1. RGD can be reduced to an effective one-dimensional recursion which mirrors the evolution of the AMP iterates. 2. From a warm start, RGD rapidly converges to a fixed point in correlation space, which recovers Bayes-optimal performance when run on the posterior. 3. Conditioned on widely believed mixing results for the SK model, we recover the phase transition for non-trivial inference.

Problem

• Relates Glauber dynamics recovery to Bayes-optimal AMP performance
• Analyzes restricted Gaussian dynamics via one-dimensional recursion mirroring AMP
• Establishes phase transition for inference using SK model mixing results

Innovation

• Relates Glauber dynamics to AMP via RGD
• Reduces RGD to one-dimensional recursion mirroring AMP
• RGD converges to Bayes-optimal fixed point from warm start