🤖 AI Summary
Stochastic MALA (sMALA) suffers from posterior drift—deviating from the true Gibbs posterior toward a non-Gibbs target—due to minibatch-induced bias, thereby compromising uncertainty quantification in Bayesian neural networks.
Method: We propose a stochastic Metropolis–Hastings algorithm with an explicit, computable correction term that rigorously restores sampling from the original Gibbs posterior. This is the first MH-based stochastic sampler to incorporate an analytically tractable bias correction.
Contributions/Results: We establish a PAC-Bayesian theoretical foundation for deep nonparametric regression, proving that the corrected algorithm achieves optimal posterior contraction rates and yields credible sets with guaranteed high coverage probability. Numerical experiments demonstrate that its uncertainty quantification performance matches classical MALA and significantly outperforms uncorrected sMALA, validating both statistical fidelity and practical efficacy.
📝 Abstract
A Metropolis-Hastings step is widely used for gradient-based Markov chain Monte Carlo methods in uncertainty quantification. By calculating acceptance probabilities on batches, a stochastic Metropolis-Hastings step saves computational costs, but reduces the effective sample size. We show that this obstacle can be avoided by a simple correction term. We study statistical properties of the resulting stationary distribution of the chain if the corrected stochastic Metropolis-Hastings approach is applied to sample from a Gibbs posterior distribution in a nonparametric regression setting. Focusing on deep neural network regression, we prove a PAC-Bayes oracle inequality which yields optimal contraction rates and we analyze the diameter and show high coverage probability of the resulting credible sets. With a numerical example in a high-dimensional parameter space, we illustrate that credible sets and contraction rates of the stochastic Metropolis-Hastings algorithm indeed behave similar to those obtained from the classical Metropolis-adjusted Langevin algorithm.
