Improving the statistical efficiency of cross-conformal prediction

2025-03-03

AI Summary

This paper addresses the critical issue of excessively wide prediction intervals in cross-conformal prediction, which severely limits practical utility. Under the strict theoretical guarantee of marginal coverage at least $1-2alpha$, we propose a novel randomized $p$-value aggregation method. Grounded in exchangeability theory, our approach introduces a randomization mechanism to optimize $p$-value combination and reformulates both the cross-conformal framework and folding strategy, thereby breaking the conventional width–coverage trade-off bottleneck. Theoretical analysis confirms tight coverage guarantees, while simulations demonstrate a 15–30% reduction in average prediction interval width, substantially improving statistical efficiency. The core innovation lies in the principled integration of randomization with exchangeability, enabling more efficient and compact conformal inference.

Technology Category

• Reasoning under Uncertainty: Stochastic Optimization
• Machine Learning: Calibration & Uncertainty Quantification
• Constraint Satisfaction and Optimization: Other Foundations of Constraint Satisfaction

Application Category

• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Security and Privacy: Large-scale security measurements
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Vovk (2015) introduced cross-conformal prediction, a modification of split conformal designed to improve the width of prediction sets. The method, when trained with a miscoverage rate equal to $alpha$ and $n gg K$, ensures a marginal coverage of at least $1 - 2alpha - 2(1-alpha)(K-1)/(n+K)$, where $n$ is the number of observations and $K$ denotes the number of folds. A simple modification of the method achieves coverage of at least $1-2alpha$. In this work, we propose new variants of both methods that yield smaller prediction sets without compromising the latter theoretical guarantee. The proposed methods are based on recent results deriving more statistically efficient combination of p-values that leverage exchangeability and randomization. Simulations confirm the theoretical findings and bring out some important tradeoffs.

Problem

• Improve statistical efficiency of cross-conformal prediction
• Reduce prediction set size without losing coverage guarantees
• Leverage exchangeability and randomization for efficient p-value combination

Innovation

• Enhances cross-conformal prediction efficiency
• Uses p-value combination techniques
• Maintains theoretical coverage guarantees