Prediction Sets for Counterfactual Decisions: Coverage, Optimality, and Conformal Prediction

2026-07-02

AI Summary

This work addresses the challenge of ensuring reliable outcome coverage under high-stakes counterfactual decision-making, where conventional uncertainty quantification methods often fail to guarantee coverage for the chosen actions, leading to suboptimal or ineffective decisions. The authors propose a novel paradigm—Policy-Coupled Coverage—that directly links prediction sets to decision actions and establish its theoretical equivalence to both general coverage guarantees and risk-averse policy optimization, yielding a globally optimal form for prediction sets. Building on this foundation, they develop a two-stage Policy-Coupled Risk-Averse Conformal Prediction (PC-RACP) algorithm that rigorously ensures finite-sample coverage. Experiments demonstrate that PC-RACP significantly improves decision utility while maintaining valid coverage, outperforming existing approaches in both simulated and real-world email marketing scenarios.

Abstract

Predictions are increasingly used to guide high-stakes decisions, from treatment selection to policy making. To ensure reliability with imperfect predictions, uncertainty quantification methods such as conformal prediction build prediction sets with coverage guarantees. However, statistical validity alone does not immediately determine the decisions to take, nor the optimality thereof. This gap is especially delicate in counterfactual settings where the outcome that materializes depends on the action taken, so uncertainty cannot be specified independently of the decision rule. We develop a decision-theoretic framework for uncertainty-informed counterfactual decisions. We identify a novel notion of \emph{policy-coupled coverage} -- namely, coverage of the realized outcome under the action induced by the prediction sets themselves -- as the optimal and lossless interface between uncertainty and action. It plays three roles. First, it justifies acting via a natural max-min rule as minimax-optimal under distributional ambiguity. Second, optimizing prediction sets under policy-coupled coverage is equivalent both to a stronger universal-coverage formulation and to the direct risk-averse optimization over policies and utility certificates; this equivalence yields the explicit form of the population-optimal prediction sets. Third, it admits a two-stage procedure, Policy-Coupled Risk-Averse Conformal Prediction (PC-RACP), that approximates these optimal sets with rigorous finite-sample coverage. Simulations and a real email-marketing experiment confirm that PC-RACP delivers higher utility than existing approaches while maintaining valid coverage, and that ignoring the counterfactual structure of the decision problem is suboptimal for both validity and utility.

Problem

• counterfactual decisions
• prediction sets
• coverage
• uncertainty quantification
• decision theory

Innovation

• policy-coupled coverage
• conformal prediction
• counterfactual decisions
• risk-averse optimization
• minimax optimality