Bridge Sampling Diagnostics

2025-08-20

AI Summary

Bayesian model selection and averaging rely on the marginal likelihood, whose exact computation is intractable for complex models; standard estimators such as bridge sampling often yield high-variance approximations. This paper proposes a diagnostic, low-overhead framework to assess the reliability of marginal likelihood estimates: it introduces Pareto-$hat{k}$ diagnostics and block reordering into the bridge sampling pipeline, enabling robust quantification of Monte Carlo standard error (MCSE) without additional posterior sampling. The method integrates bridge sampling, MCSE estimation, and a dual-diagnostic mechanism. In simulation studies and real-world posterior distributions from posteriordb, it substantially reduces estimator variability and enhances credibility. The resulting tool provides a reproducible, verifiable, and practical solution for Bayesian model comparison.

Technology Category

• Reasoning under Uncertainty: Relational Probabilistic Models
• Machine Learning: Calibration & Uncertainty Quantification
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs

Abstract

In Bayesian statistics, the marginal likelihood is used for model selection and averaging, yet it is often challenging to compute accurately for complex models. Approaches such as bridge sampling, while effective, may suffer from issues of high variability of the estimates. We present how to estimate Monte Carlo standard error (MCSE) for bridge sampling, and how to diagnose the reliability of MCSE estimates using Pareto-$hat{k}$ and block reshuffling diagnostics without the need to repeatedly re-run full posterior inference. We demonstrate the behavior with increasingly more difficult simulated posteriors and many real posteriors from the posteriordb database.

Problem

• Estimating marginal likelihood for Bayesian model selection
• Assessing bridge sampling variability and estimate reliability
• Diagnosing Monte Carlo standard error without repeated inference

Innovation

• Estimating MCSE for bridge sampling
• Diagnosing reliability with Pareto-hat{k}
• Using block reshuffling without re-running inference