Sampling from the Continuous Random Energy Model in Total Variation Distance

2024-07-01

arXiv.org

AI Summary

This paper addresses approximate sampling from the Gibbs distribution of the Continuous Random Energy Model (CREM) in the high-temperature regime (β < βₘᵢₙ), overcoming the prior limitation—under KL divergence—of only sublinear accuracy (o(N)). It achieves, for the first time, polynomial-time approximate sampling with provable guarantees in total variation (TV) distance. Two hybrid algorithms are proposed, combining MCMC with sequential sampling; their runtime is rigorously characterized: polynomial in the inverse TV accuracy and failure probability, and exponential in (1/g)ᴼ⁽¹⁾. Key contributions include: (i) a novel compatibility theory for “tilted” CREMs; (ii) quantitative bounds on partition function fluctuations; (iii) proof that the spectral gap is inherently algebraically small, necessitating algebraic dependence in runtime; and (iv) a sharp separation between the static phase transition point β_c (in the A-concave case) and the algorithmic threshold β_G (in the non-concave case), with strict superiority over existing KL-based approaches.

Technology Category

• Machine Learning: Probabilistic Circuits and Graphical Models
• Reasoning under Uncertainty: Relational Probabilistic Models
• Search and Optimization: Mixed Discrete/Continuous Search

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Security and Privacy: Large-scale security measurements
• Web Mining and Content Analysis: Models for Web evolution

Abstract

The continuous random energy model (CREM) is a toy model of spin glasses on ${0,1}^N$ that, in the limit, exhibits an infinitely hierarchical correlation structure. We give two polynomial-time algorithms to approximately sample from the Gibbs distribution of the CREM in the high-temperature regime $eta<eta_{min}:=min{eta_c,eta_G}$, based on a Markov chain and a sequential sampler. The running time depends algebraically on the desired TV distance and failure probability and exponentially in $(1/g)^{O(1)}$, where $g$ is the gap to a certain inverse temperature threshold $eta_{min}$; this contrasts with previous results which only attain $o(N)$ accuracy in KL divergence. If the covariance function $A$ of the CREM is concave, the algorithms work up to the critical threshold $eta_c$, which is the static phase transition point; while for $A$ non-concave, if $eta_G<eta_c$, the algorithms work up to the known algorithmic threshold $eta_G$ proposed in Addario-Berry and Maillard (2020) for non-trivial sampling guarantees. Our result depends on quantitative bounds for the fluctuation of the partition function and a new contiguity result of the ``tilted"CREM obtained from sampling, which is of independent interest. We also show that the spectral gap is exponentially small with high probability, suggesting that the algebraic dependence is unavoidable with a Markov chain approach.

Problem

• Sampling from the CREM in high-temperature regime.
• Developing polynomial-time algorithms for Gibbs distribution.
• Analyzing spectral gap and algorithmic thresholds.

Innovation

• Polynomial-time sampling algorithms
• Markov chain and sequential sampler
• Algebraic running time dependence