Separation-Utility Pareto Frontier: An Information-Theoretic Characterization

2026-02-04

AI Summary

This work investigates the optimal trade-off between predictive utility and separation fairness—defined as the conditional independence of predictions from sensitive attributes given the true label—in machine learning. Leveraging information-theoretic tools, the study rigorously characterizes the Pareto frontier between these objectives for the first time, proving its concavity and demonstrating that the marginal cost of enhancing separation fairness increases with utility. Building on this insight, the authors propose a general, differentiable, and theoretically grounded regularizer based on conditional mutual information, which can be seamlessly integrated into any deep learning model to enforce fairness constraints. Experiments on COMPAS, UCI Adult, UCI Bank, and CelebA datasets show that the method substantially reduces violations of separation fairness while maintaining or surpassing the predictive performance of existing baselines.

Technology Category

• Machine Learning: Ethics, Bias, and Fairness
• Computer Vision: Bias, Fairness & Privacy
• Philosophy and Ethics of AI: Bias, Fairness & Equity

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Fairness, privacy, and diversity in economic environments

Abstract

We study the Pareto frontier (optimal trade-off) between utility and separation, a fairness criterion requiring predictive independence from sensitive attributes conditional on the true outcome. Through an information-theoretic lens, we prove a characterization of the utility-separation Pareto frontier, establish its concavity, and thereby prove the increasing marginal cost of separation in terms of utility. In addition, we characterize the conditions under which this trade-off becomes strict, providing a guide for trade-off selection in practice. Based on the theoretical characterization, we develop an empirical regularizer based on conditional mutual information (CMI) between predictions and sensitive attributes given the true outcome. The CMI regularizer is compatible with any deep model trained via gradient-based optimization and serves as a scalar monitor of residual separation violations, offering tractable guarantees during training. Finally, numerical experiments support our theoretical findings: across COMPAS, UCI Adult, UCI Bank, and CelebA, the proposed method substantially reduces separation violations while matching or exceeding the utility of established baseline methods. This study thus offers a provable, stable, and flexible approach to enforcing separation in deep learning.

Problem

• separation
• utility
• fairness
• Pareto frontier
• conditional independence

Innovation

• Pareto frontier
• separation fairness
• conditional mutual information
• information-theoretic characterization
• deep learning regularizer