E-Values Expand the Scope of Conformal Prediction

2025-03-17

AI Summary

Traditional conformal prediction relies on p-values, distributional assumptions, and rank-based testing frameworks, limiting its applicability under weak supervision or unknown data distributions. To address this, we propose *conformal e-prediction*, a novel paradigm grounded in e-values—nonnegative random variables with expectation at most one under the null. This framework requires only exchangeability, enabling distribution-free uncertainty quantification valid at any time point. Our key contributions are: (1) the first integration of e-values into conformal prediction, supporting both batch online inference and data-dependent fixed-size prediction sets; (2) robustness under fuzzy ground-truth settings; and (3) anytime-valid inference with a data-driven coverage control algorithm that guarantees finite-sample validity of marginal coverage. By eliminating reliance on p-values and distributional assumptions, our approach significantly broadens the scope of conformal prediction to settings with limited supervision or unknown underlying distributions, striking a new balance between theoretical rigor and practical flexibility.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization

Application Category

• Security and Privacy: Data transparency and provenance
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

Conformal prediction is a powerful framework for distribution-free uncertainty quantification. The standard approach to conformal prediction relies on comparing the ranks of prediction scores: under exchangeability, the rank of a future test point cannot be too extreme relative to a calibration set. This rank-based method can be reformulated in terms of p-values. In this paper, we explore an alternative approach based on e-values, known as conformal e-prediction. E-values offer key advantages that cannot be achieved with p-values, enabling new theoretical and practical capabilities. In particular, we present three applications that leverage the unique strengths of e-values: batch anytime-valid conformal prediction, fixed-size conformal sets with data-dependent coverage, and conformal prediction under ambiguous ground truth. Overall, these examples demonstrate that e-value-based constructions provide a flexible expansion of the toolbox of conformal prediction.

Problem

• Expands conformal prediction using e-values for uncertainty quantification.
• Enables batch anytime-valid prediction and data-dependent coverage sets.
• Addresses conformal prediction under ambiguous ground truth scenarios.

Innovation

• Uses e-values for conformal prediction
• Enables batch anytime-valid prediction
• Supports data-dependent coverage sets