Multi-Variable Conformal Prediction: Optimizing Prediction Sets without Data Splitting

2026-05-12

AI Summary

Traditional conformal prediction relies on scalar scores and a single threshold, requiring data splitting, which results in fixed prediction set shapes and limited efficiency. This work proposes a Multivariate Conformal Prediction (MCP) framework that, for the first time, integrates scenario theory into conformal prediction. By employing vector-valued scoring functions and multiple calibration variables, MCP unifies prediction set design and calibration into an end-to-end optimization problem, enabling joint shape optimization without data splitting while guaranteeing valid coverage. Two variants—RemMCP and RelMCP—are further introduced, based on constraint removal and relaxation strategies, respectively, to accommodate convex and non-convex scoring functions. Experiments demonstrate that MCP achieves significantly smaller or comparable prediction set sizes under the target coverage, while substantially reducing variance across calibration runs.

Abstract

Conformal prediction constructs prediction sets with finite-sample coverage guarantees, but its calibration stage is structurally constrained to a scalar score function and a single threshold variable - forcing shapes of prediction sets to be fixed before calibration, typically through data splitting. We introduce multi-variable conformal prediction (MCP), a framework that extends conformal prediction to vector-valued score functions with multiple simultaneous calibration variables. Building on scenario theory as a principled framework for certifying data-driven decisions, MCP unifies prediction set design and calibration into a single optimization problem, eliminating data splitting without sacrificing coverage guarantees. We propose two computationally efficient variants: RemMCP, grounded in constrained optimization with constraint removal, which admits a clean generalization of split conformal prediction; and RelMCP, based on iterative optimization with constraint relaxation, which supports non-convex score functions at the cost of possibly greater conservatism. Through numerical experiments on ellipsoidal and multi-modal prediction sets, we demonstrate that RemMCP and RelMCP consistently meet the target coverage with prediction set sizes smaller than or comparable to those of baselines with data split, while considerably reducing variance across calibration runs - a direct consequence of using all available data for shape optimization and calibration simultaneously.

Problem

• Conformal Prediction
• Prediction Sets
• Data Splitting
• Score Function
• Coverage Guarantee

Innovation

• Conformal Prediction
• Multi-Variable Calibration
• Scenario Theory
• Data Splitting Elimination
• Prediction Set Optimization