Counterfactually Fair Conformal Prediction

2025-10-09

AI Summary

This paper addresses the lack of fairness guarantees in conformal prediction by proposing CF-CP, the first framework for constructing counterfactually fair prediction sets. The core challenge lies in ensuring marginal coverage—i.e., guaranteeing that the prediction set contains the true label with probability at least $1-alpha$—while eliminating influence of protected attributes (e.g., gender, race) on uncertainty quantification. CF-CP achieves this by defining and enforcing distributional symmetry of conformity scores under counterfactual interventions, without modifying or retraining the underlying predictive model. Thus, it simultaneously attains statistical validity and counterfactual fairness. Experiments across synthetic and multiple real-world datasets demonstrate CF-CP’s effectiveness for both classification and regression tasks: it reliably achieves the target coverage level while incurring only negligible inflation in prediction set size.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Computer Vision: Bias, Fairness & Privacy
• Reasoning under Uncertainty: Causality

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Fairness, privacy, and diversity in economic environments

Abstract

While counterfactual fairness of point predictors is well studied, its extension to prediction sets--central to fair decision-making under uncertainty--remains underexplored. On the other hand, conformal prediction (CP) provides efficient, distribution-free, finite-sample valid prediction sets, yet does not ensure counterfactual fairness. We close this gap by developing Counterfactually Fair Conformal Prediction (CF-CP) that produces counterfactually fair prediction sets. Through symmetrization of conformity scores across protected-attribute interventions, we prove that CF-CP results in counterfactually fair prediction sets while maintaining the marginal coverage property. Furthermore, we empirically demonstrate that on both synthetic and real datasets, across regression and classification tasks, CF-CP achieves the desired counterfactual fairness and meets the target coverage rate with minimal increase in prediction set size. CF-CP offers a simple, training-free route to counterfactually fair uncertainty quantification.

Problem

• Extends counterfactual fairness from point predictors to prediction sets
• Ensures prediction sets maintain fairness under protected-attribute interventions
• Achieves counterfactual fairness while preserving marginal coverage guarantees

Innovation

• Symmetrizing conformity scores across protected attributes
• Maintaining marginal coverage in fair prediction sets
• Providing training-free fair uncertainty quantification