Bernstein-von Mises for Adaptively Collected Data

2025-11-10

AI Summary

This paper investigates the asymptotic equivalence between Bayesian and frequentist uncertainty quantification (UQ) under adaptive data collection—such as in multi-armed bandits and reinforcement learning—where observations are neither independent nor identically distributed (i.i.d.). Classical Bernstein–von Mises (BvM) theorems fail here due to dependence induced by adaptive sampling. We extend the BvM theorem to general adaptive settings, proving that the posterior distribution remains asymptotically normal under weakened stability conditions, and that Bayesian credible intervals and Wald-type confidence intervals achieve asymptotic equivalence. Crucially, we uncover a counterintuitive phenomenon: “prior vanishing without frequentist validity”—when stability conditions break down, the posterior concentrates, yet Bayesian UQ loses its frequentist coverage guarantee. Our theoretical findings are established via rigorous asymptotic analysis and validated through extensive simulations.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty
• Game Theory and Economic Paradigms: Adversarial Learning

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Security and Privacy: Large-scale security measurements
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Uncertainty quantification (UQ) for adaptively collected data, such as that coming from adaptive experiments, bandits, or reinforcement learning, is necessary for critical elements of data collection such as ensuring safety and conducting after-study inference. The data's adaptivity creates significant challenges for frequentist UQ, yet Bayesian UQ remains the same as if the data were independent and identically distributed (i.i.d.), making it an appealing and commonly used approach. Bayesian UQ requires the (correct) specification of a prior distribution while frequentist UQ does not, but for i.i.d. data the celebrated Bernstein-von Mises theorem shows that as the sample size grows, the prior'washes out'and Bayesian UQ becomes frequentist-valid, implying that the choice of prior need not be a major impediment to Bayesian UQ as it makes no difference asymptotically. This paper for the first time extends the Bernstein-von Mises theorem to adaptively collected data, proving asymptotic equivalence between Bayesian UQ and Wald-type frequentist UQ in this challenging setting. Our result showing this asymptotic agreement does not require the standard stability condition required by works studying validity of Wald-type frequentist UQ; in cases where stability is satisfied, our results combined with these prior studies of frequentist UQ imply frequentist validity of Bayesian UQ. Counterintuitively however, they also provide a negative result that Bayesian UQ is not asymptotically frequentist valid when stability fails, despite the fact that the prior washes out and Bayesian UQ asymptotically matches standard Wald-type frequentist UQ. We empirically validate our theory (positive and negative) via a range of simulations.

Problem

• Extends Bernstein-von Mises theorem to adaptively collected data
• Proves asymptotic equivalence between Bayesian and frequentist uncertainty quantification
• Shows Bayesian UQ validity depends on stability conditions in adaptive settings

Innovation

• Extends Bernstein-von Mises theorem to adaptive data
• Proves Bayesian and frequentist UQ asymptotic equivalence
• Reveals Bayesian UQ validity depends on stability condition