Variational predictive resampling

2026-05-11

AI Summary

This work addresses the limitations of traditional variational inference—such as mean-field approximations, which often fail to capture posterior dependencies and thus yield inaccurate uncertainty quantification—while avoiding the prohibitive computational cost of Markov chain Monte Carlo (MCMC). The authors propose Variational Predictive Resampling (VPR), a novel approach that integrates variational inference within a predictive resampling framework. By iteratively simulating future observations, updating the variational approximation, and recording the corresponding parameters, VPR efficiently approximates the full Bayesian posterior. The method retains scalability while recovering posterior dependencies omitted by mean-field assumptions and is theoretically shown to converge to the true posterior in Gaussian location models. Empirical results demonstrate that VPR significantly improves the accuracy of posterior uncertainty quantification across multiple models, achieving computational efficiency comparable to or better than MCMC.

Abstract

Bayesian inference provides principled uncertainty quantification, but accurate posterior sampling with MCMC can be computationally prohibitive for modern applications. Variational inference (VI) offers a scalable alternative and often yields accurate predictive distributions, but cheap variational families such as mean-field (MF) can produce over-concentrated approximations that miss posterior dependence. We propose variational predictive resampling (VPR), a scalable posterior sampling method that exploits VI's predictive strength within a predictive-resampling framework to better approximate the Bayesian posterior. Given a prior--likelihood pair, VPR repeatedly imputes future observations from the current variational predictive, updates the variational approximation after each imputation, and records the parameter value implied by the completed sample. We establish conditions under which the law of the parameter returned by VPR is well defined and show that its finite-horizon approximation converges to this limit.In a tractable Gaussian location model, we show that VPR with MF variational predictives converges to the exact Bayesian posterior, whereas the optimal MF-VI approximation retains a non-vanishing asymptotic gap. Experiments on linear regression, logistic regression, and hierarchical linear mixed-effects models demonstrate that VPR substantially improves posterior uncertainty quantification and recovers posterior dependence missed by MF-VI, while remaining computationally competitive with, and often more efficient than, MCMC.

Problem

• Bayesian inference
• posterior sampling
• variational inference
• mean-field approximation
• uncertainty quantification

Innovation

• variational predictive resampling
• Bayesian inference
• posterior dependence
• scalable sampling
• mean-field variational inference