Robust Bayes-Assisted Conformal Prediction

2026-07-05

AI Summary

This work addresses the significant inflation of prediction sets in Bayesian-assisted conformal prediction when the prior is misspecified, which undermines predictive efficiency. To mitigate this issue, the authors propose the RoBAS framework, which constructs robust nonconformity scores adaptive to prior quality—enhancing efficiency when the prior is reliable and automatically reverting to a robust baseline when it is not. The approach innovatively introduces two novel score functions grounded in heavy-tailed Bayesian working models and closed-form empirical Bayes shrinkage, effectively integrating Bayesian modeling, conformal prediction, and the Distance-to-Average benchmark. Empirical results demonstrate that RoBAS substantially narrows prediction interval widths under distributional shift while maintaining coverage performance comparable to existing methods in the absence of such shifts.

Abstract

Bayes-assisted conformal prediction combines the strengths of Bayesian modelling with exact, distribution-free frequentist coverage guarantees. Although conformal validity is preserved even when the Bayesian working model (BWM) is misspecified, the size of the resulting prediction sets can degrade substantially when the prior is poorly aligned with the observed data. We address this limitation by introducing RoBAS (Robust Bayes-Assisted Shrinkage): a Bayes-assisted framework for constructing robust nonconformity scores, with two instantiations: one induced by a heavy-tailed BWM, and a closed-form empirical Bayes shrinkage score. The resulting scores adapt to the quality of the working information encoded in the prior: when this information is reliable, they exploit it to produce efficient prediction sets; when it is weak or inaccurate, they revert to the Distance-To-Average (DTA) score, a robust non-informative baseline. We evaluate the proposed scores on tabular and image regression tasks where the training distribution may differ from the calibration and test distributions, while the calibration and test data themselves remain exchangeable. We find that they are competitive with widely used scores in the absence of such shift, while substantially reducing interval widths in shifted settings.

Problem

• conformal prediction
• Bayesian prior misspecification
• prediction set size
• distribution shift
• nonconformity scores

Innovation

• Robust Conformal Prediction
• Bayes-Assisted Shrinkage
• Nonconformity Score
• Distribution Shift
• Empirical Bayes