On the Low-Temperature MCMC threshold: the cases of sparse tensor PCA, sparse regression, and a geometric rule

2024-08-01

arXiv.org

AI Summary

This work investigates the posterior maximization capability of low-temperature MCMC in sparse tensor PCA and sparse regression, aiming to characterize the theoretical threshold for polynomial-time solvability. Method: We develop the first general, verifiable criterion for polynomial convergence of low-temperature Metropolis chains and integrate it with the Overlap Gap Property (OGP) framework. Contribution/Results: We rigorously establish the tightness of this criterion, thereby identifying—for the first time—the precise “low-temperature local MCMC solvability threshold” for both models. Below this threshold, low-temperature MCMC provably fails to reach the maximum a posteriori (MAP) estimator within polynomial time; its computational power is fundamentally weaker than that of efficient low-degree polynomial algorithms. Our results provide a key theoretical characterization of the computational limits of MCMC in high-dimensional sparse inference, along with a practical, verifiable tool for diagnosing such limits.

Technology Category

• Machine Learning: Probabilistic Circuits and Graphical Models
• Reasoning under Uncertainty: Graphical Models
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Security and Privacy: Large-scale security measurements
• Web Mining and Content Analysis: Models for Web evolution

Abstract

Over the last years, there has been a significant amount of work studying the power of specific classes of computationally efficient estimators for multiple statistical parametric estimation tasks, including the estimators classes of low-degree polynomials, spectral methods, and others. Despite that, our understanding of the important class of MCMC methods remains quite poorly understood. For instance, for many models of interest, the performance of even zero-temperature (greedy-like) MCMC methods that simply maximize the posterior remains elusive. In this work, we provide an easy to check condition under which the low-temperature Metropolis chain maximizes the posterior in polynomial-time with high probability. The result is generally applicable, and in this work, we use it to derive positive MCMC results for two classical sparse estimation tasks: the sparse tensor PCA model and sparse regression. Interestingly, in both cases, we also leverage the Overlap Gap Property framework for inference (Gamarnik, Zadik AoS '22) to prove that our results are tight: no low-temperature local MCMC method can achieve better performance. In particular, our work identifies the"low-temperature (local) MCMC threshold"for both sparse models. Interestingly, in the sparse tensor PCA model our results indicate that low-temperature local MCMC methods significantly underperform compared to other studied time-efficient methods, such as the class of low-degree polynomials.

Problem

• Understanding MCMC methods' performance in sparse models
• Identifying low-temperature MCMC thresholds for sparse tasks
• Comparing MCMC efficiency with other time-efficient estimators

Innovation

• Low-temperature Metropolis chain maximizes posterior
• Overlap Gap Property ensures tight results
• Identifies MCMC thresholds for sparse models