Weighted Bayesian Conformal Prediction

2026-04-07

AI Summary

This work addresses the limitations of existing Bayesian conformal prediction methods, which rely on the i.i.d. assumption and struggle under distribution shift, and weighted conformal prediction approaches, which handle shift but lack principled Bayesian uncertainty quantification. The paper introduces the first extension of Bayesian conformal prediction to importance-weighted settings by proposing a weighted Dirichlet prior based on the Kish effective sample size, thereby integrating Bayesian inference with weighting mechanisms. Theoretically, the method establishes four key guarantees: variance matching, posterior convergence rate, improved conditional coverage, and data-dependent coverage validity. Empirical evaluations on both synthetic and real-world spatial datasets demonstrate that the proposed Weighted Bayesian Conformal Prediction (WBCP) significantly enhances the richness and interpretability of uncertainty representations while maintaining rigorous coverage guarantees.

Abstract

Conformal prediction provides distribution-free prediction intervals with finite-sample coverage guarantees, and recent work by Snell \& Griffiths reframes it as Bayesian Quadrature (BQ-CP), yielding powerful data-conditional guarantees via Dirichlet posteriors over thresholds. However, BQ-CP fundamentally requires the i.i.d. assumption -- a limitation the authors themselves identify. Meanwhile, weighted conformal prediction handles distribution shift via importance weights but remains frequentist, producing only point-estimate thresholds. We propose \textbf{Weighted Bayesian Conformal Prediction (WBCP)}, which generalizes BQ-CP to arbitrary importance-weighted settings by replacing the uniform Dirichlet $\Dir(1,\ldots,1)$ with a weighted Dirichlet $\Dir(\neff \cdot \tilde{w}_1, \ldots, \neff \cdot \tilde{w}_n)$, where $\neff$ is Kish's effective sample size. We prove four theoretical results: (1)~$\neff$ is the unique concentration parameter matching frequentist and Bayesian variances; (2)~posterior standard deviation decays as $O(1/\sqrt{\neff})$; (3)~BQ-CP's stochastic dominance guarantee extends to per-weight-profile data-conditional guarantees; (4)~the HPD threshold provides $O(1/\sqrt{\neff})$ improvement in conditional coverage. We instantiate WBCP for spatial prediction as \emph{Geographical BQ-CP}, where kernel-based spatial weights yield per-location posteriors with interpretable diagnostics. Experiments on synthetic and real-world spatial datasets demonstrate that WBCP maintains coverage guarantees while providing substantially richer uncertainty information.

Problem

• conformal prediction
• distribution shift
• Bayesian uncertainty
• importance weighting
• coverage guarantee

Innovation

• Weighted Bayesian Conformal Prediction
• Dirichlet posterior
• distribution shift
• effective sample size
• spatial prediction