Counterfactual Fairness by Combining Factual and Counterfactual Predictions

2024-09-03

arXiv.org

AI Summary

In high-stakes domains such as healthcare and hiring, achieving counterfactual fairness (CF) often conflicts with maintaining predictive accuracy—a fundamental trade-off previously uncharacterized in a model-agnostic manner. Method: We propose the “Pareto-optimal Fairness” framework, which enforces CF without degrading optimal prediction performance. Our approach integrates counterfactual reasoning, causal modeling, and optimal predictor refinement, and accommodates incomplete causal knowledge. Contribution/Results: We provide the first theoretical characterization of the inherent fairness–accuracy trade-off, rigorously quantifying its boundary. Experiments on synthetic and semi-synthetic datasets demonstrate that our method substantially improves CF guarantees while preserving—and in many cases surpassing—the predictive accuracy of baseline models. This achieves synergistic optimization of fairness and accuracy, advancing practical deployment of fair machine learning in critical applications.

Technology Category

• Machine Learning: Causal Learning
• Game Theory and Economic Paradigms: Fair Division
• Reasoning under Uncertainty: Causality

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Fairness, privacy, and diversity in economic environments

Abstract

In high-stake domains such as healthcare and hiring, the role of machine learning (ML) in decision-making raises significant fairness concerns. This work focuses on Counterfactual Fairness (CF), which posits that an ML model's outcome on any individual should remain unchanged if they had belonged to a different demographic group. Previous works have proposed methods that guarantee CF. Notwithstanding, their effects on the model's predictive performance remains largely unclear. To fill in this gap, we provide a theoretical study on the inherent trade-off between CF and predictive performance in a model-agnostic manner. We first propose a simple but effective method to cast an optimal but potentially unfair predictor into a fair one without losing the optimality. By analyzing its excess risk in order to achieve CF, we quantify this inherent trade-off. Further analysis on our method's performance with access to only incomplete causal knowledge is also conducted. Built upon it, we propose a performant algorithm that can be applied in such scenarios. Experiments on both synthetic and semi-synthetic datasets demonstrate the validity of our analysis and methods.

Problem

• Counterfactual Fairness
• Predictive Accuracy
• Bias Mitigation

Innovation

• Counterfactual Fairness
• Machine Learning Ethics
• Imperfect Information