Generalized Conformal Predictive Systems Under Distributional Shifts

2026-06-09

AI Summary

This work addresses the challenge of maintaining calibration and validity in conformal prediction under non-exchangeable distribution shifts. It presents the first extension of generalized conformal prediction to this setting by introducing observation-specific permutation weights to model distributional shift and constructing a robust prediction envelope via uncertainty sets over these weights, thereby guaranteeing valid coverage either in finite samples or asymptotically. The method integrates weighted permutations, conformal scores, binning, and isotonic distribution regression to achieve computational efficiency. Experiments on covariate shift and feedback-driven biomolecular design tasks demonstrate that the resulting prediction bands adaptively widen with increasing distribution shift and effectively tighten as sample size grows, exhibiting both reliability and adaptivity.

Abstract

Conformal predictive systems (CPS) output calibrated bands of CDFs under exchangeability. We extend generalized CPS to non-exchangeable settings by encoding distributional shifts through observation-specific permutation weights. This yields shift-aware predictive systems that remain valid whenever the test point is, conditionally on the unordered sample, a weighted draw from the observed atoms. Since such weights are typically estimated, we introduce weight-uncertainty boxes and construct robust CPS envelopes with finite-sample or asymptotic confidence guarantees. We derive efficient computation for conformity-measure CPS, conformal binning, and conformal isotonic distributional regression. Experiments under covariate shift and feedback-driven biomolecular design show calibrated predictive bands that widen under stronger shifts and tighten as sample size increases.

Problem

• conformal prediction
• distributional shift
• non-exchangeability
• predictive uncertainty
• calibration

Innovation

• Conformal Predictive Systems
• Distributional Shifts
• Permutation Weights
• Weight-Uncertainty Boxes
• Calibrated Prediction Bands