Sampling from Constrained Gibbs Measures: with Applications to High-Dimensional Bayesian Inference

2026-02-25

AI Summary

This work addresses the challenge of sampling from low-temperature Gibbs distributions whose support is constrained and whose modes may lie on the boundary of the domain—regions where the Laplace approximation breaks down. In this pre-asymptotic regime, the authors uncover a local product structure: the target distribution can be decomposed as a perturbed product of a strongly log-concave distribution over the regular interior and a one-dimensional exponential-family distribution capturing the non-regular boundary behavior. Leveraging this insight, they develop an efficient sampling algorithm that integrates Langevin dynamics and, for the first time, provide non-asymptotic sampling error guarantees for models with boundary modes. The method demonstrates both theoretical rigor and practical efficiency across several Bayesian inference tasks, including high-dimensional logistic regression, Poisson linear models, and Gaussian mixture models.

Technology Category

• Machine Learning: Probabilistic Circuits and Graphical Models
• Reasoning under Uncertainty: Relational Probabilistic Models
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments

Abstract

This paper considers a non-standard problem of generating samples from a low-temperature Gibbs distribution with \emph{constrained} support, when some of the coordinates of the mode lie on the boundary. These coordinates are referred to as the non-regular part of the model. We show that in a ``pre-asymptotic'' regime in which the limiting Laplace approximation is not yet valid, the low-temperature Gibbs distribution concentrates on a neighborhood of its mode. Within this region, the distribution is a bounded perturbation of a product measure: a strongly log-concave distribution in the regular part and a one-dimensional exponential-type distribution in each coordinate of the non-regular part. Leveraging this structure, we provide a non-asymptotic sampling guarantee by analyzing the spectral gap of Langevin dynamics. Key examples of low-temperature Gibbs distributions include Bayesian posteriors, and we demonstrate our results on three canonical examples: a high-dimensional logistic regression model, a Poisson linear model, and a Gaussian mixture model.

Problem

• constrained Gibbs measures
• low-temperature sampling
• non-regular mode
• boundary constraints
• Bayesian inference

Innovation

• constrained Gibbs measures
• non-regular posterior
• low-temperature sampling
• Langevin dynamics
• spectral gap analysis