Facets of Disparate Impact: Evaluating Legally Consistent Bias in Machine Learning

2024-10-21

International Conference on Information and Knowledge Management

AI Summary

This paper addresses the misalignment between algorithmic bias assessment and legal standards by proposing a quantification framework rigorously grounded in U.S. anti-discrimination law. Methodologically, it distinguishes legally salient discriminatory testing from systemic disparity through legal contextualization, and introduces the Objective Fairness Index (OFI)—a metric integrating objective test theory and measurement stability, using marginal benefit as a proxy to quantify legal compliance of algorithmic decisions. Its key contribution lies in being the first fairness metric to embed legal admissibility directly into its design, enabling a paradigm shift in algorithmic auditing from statistical fairness to legally grounded fairness. Empirical evaluation on real-world judicial prediction systems—including COMPAS—demonstrates that OFI reliably detects unlawful discrimination, offering regulators and auditors the first quantitative tool with both legal interpretability and operational utility.

Technology Category

• Philosophy and Ethics of AI: Bias, Fairness & Equity
• Machine Learning: Ethics, Bias, and Fairness
• Computer Vision: Bias, Fairness & Privacy

Application Category

• Responsible Web: Algorithmic accountability and transparency on the web
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Social Networks and Social Media: Fairness and bias in social network and social media analysis

Abstract

Leveraging current legal standards, we define bias through the lens of marginal benefits and objective testing with the novel metric"Objective Fairness Index". This index combines the contextual nuances of objective testing with metric stability, providing a legally consistent and reliable measure. Utilizing the Objective Fairness Index, we provide fresh insights into sensitive machine learning applications, such as COMPAS (recidivism prediction), highlighting the metric's practical and theoretical significance. The Objective Fairness Index allows one to differentiate between discriminatory tests and systemic disparities.

Problem

• Defining legally consistent bias in machine learning
• Introducing the Objective Fairness Index metric
• Evaluating bias in sensitive applications like COMPAS

Innovation

• Defines bias using marginal benefits and objective testing
• Introduces novel metric 'Objective Fairness Index'
• Evaluates bias in sensitive ML applications like COMPAS