Post-Processing Posterior Predictive P-values

2026-05-22

AI Summary

This work addresses the non-uniform distribution of posterior predictive p-values (ppp) under the Bayesian framework, which hinders reliable model diagnostics and cross-model comparisons. The authors propose a natural calibration method that transforms ppp values into calibrated posterior predictive p-values (cppp), which follow a standard uniform distribution under the true model. This calibration establishes, for the first time, a unified and comparable scale for ppp-based assessments. The approach is grounded in a double-simulation computational framework that seamlessly integrates Bayesian inference with posterior predictive checking, and it is applicable to both parametric and nonparametric models. Theoretical analysis demonstrates favorable statistical properties of cppp, while empirical studies illustrate its effectiveness in enabling fair comparisons among models and prior specifications on real-world data.

Abstract

This article addresses issues of model criticism and model comparison in Bayesian contexts, and focusses on the use of the so-called posterior predictive p-values (ppp values). These involve a general discrepancy or conflict measure and depend on the prior, the model, and the data. They are used in statistical practice to quantify the degree of surprise or conflict in data, and for purposes of comparing different combinations of prior and model. The distribution of such ppp values is however far from uniform, as we demonstrate for different models, making their interpretation and comparison a difficult matter. We propose a natural calibration of the ppp values, where the resulting cppp values are uniform on the unit interval under model conditions. The cppp values, which in general rely on a double simulation scheme for their computation, may then be used to assess and compare different priors and models. Our methods also make it possible to compare parametric with nonparametric model specifications, in that genuine `measures of surprise' are put on the same canonical uniform scale. Our techniques are illustrated for some applications to real data. We also present supplementing theoretical results on various properties of the ppp and cppp.

Problem

• posterior predictive p-values
• model criticism
• model comparison
• Bayesian inference
• calibration

Innovation

• posterior predictive p-values
• calibration
• model criticism
• Bayesian model comparison
• uniformity