An Order of Magnitude Time Complexity Reduction for Gaussian Graphical Model Posterior Sampling Using a Reverse Telescoping Block Decomposition

2025-09-30

AI Summary

For fully Bayesian inference in high-dimensional sparse Gaussian graphical models with element-wise graph priors (e.g., graphical horseshoe), existing MCMC methods incur per-iteration complexity of O(p⁴), rendering them infeasible in p ≫ n regimes. Method: We propose a reparameterized MCMC framework that, for the first time, introduces reverse nested block decomposition to enable exact posterior sampling under non-conjugate graph priors—without approximation. This reduces per-iteration complexity to O(p³), matching that of conjugate Wishart-based methods. Contribution/Results: The method enjoys theoretical posterior consistency and demonstrates correctness, scalability, and computational efficiency on synthetic and breast cancer data—achieving an order-of-magnitude speedup over baselines. Its core innovation lies in the synergistic design of reverse structural decomposition and reparameterization, yielding the first MCMC solution for high-dimensional graphical models that simultaneously ensures statistical accuracy, scalability, and theoretical guarantees.

Technology Category

• Machine Learning: Probabilistic Circuits and Graphical Models
• Reasoning under Uncertainty: Graphical Models
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems

Abstract

We consider the problem of fully Bayesian posterior estimation and uncertainty quantification in undirected Gaussian graphical models via Markov chain Monte Carlo (MCMC) under recently-developed element-wise graphical priors, such as the graphical horseshoe. Unlike the conjugate Wishart family, these priors are non-conjugate; but have the advantage that they naturally allow one to encode a prior belief of sparsity in the off-diagonal elements of the precision matrix, without imposing a structure on the entire matrix. Unfortunately, for a graph with $p$ nodes and with $n$ samples, the state-of-the-art MCMC approaches for the element-wise priors achieve a per iteration complexity of $O(p^4),$ which is prohibitive when $pgg n$. In this regime, we develop a suitably reparameterized MCMC with per iteration complexity of $O(p^3)$, providing a one-order of magnitude improvement, and consequently bringing the computational cost at par with the conjugate Wishart family, which is also $O(p^3)$ due to a use of the classical Bartlett decomposition, but this decomposition does not apply outside the Wishart family. Importantly, the proposed benefit is obtained solely due to our reparameterization in an MCMC scheme targeting the true posterior, that reverses the recently developed telescoping block decomposition of Bhadra et al. (2024), in a suitable sense. There is no variational or any other approximate Bayesian computation scheme considered in this paper that compromises targeting the true posterior. Simulations and the analysis of a breast cancer data set confirm both the correctness and better algorithmic scaling of the proposed reverse telescoping sampler.

Problem

• Reducing time complexity for Gaussian graphical model sampling
• Enabling efficient Bayesian posterior estimation in sparse models
• Improving MCMC scalability for non-conjugate graphical priors

Innovation

• Reverse telescoping block decomposition reparameterizes MCMC
• Achieves O(p^3) per iteration complexity for posterior sampling
• Maintains true posterior targeting without variational approximation