Mutatis Mutandis: Revisiting the Comparator in Discrimination Testing

2024-05-22

arXiv.org

AI Summary

Existing fairness testing relies on simplistic similarity-based comparators, failing to adequately model how protected attributes systematically influence non-protected attributes under real-world constraints. Method: We propose a causal-driven dual-comparator framework comprising an idealized “ceteris paribus” (CP) comparator and a realistic “mutatis mutandis” (MM) comparator. The MM comparator is formalized as a counterfactual individual whose non-protected attributes co-vary under intervention on protected attributes, synthesized via integrated causal inference, generative models (GANs/VAEs), and supervised learning. Contribution/Results: This work is the first to formally distinguish CP and MM comparators. Empirical evaluation demonstrates that MM-based testing detects structural discrimination missed by CP-based methods, achieving significantly higher sensitivity, fidelity, and causal interpretability in fairness auditing.

Technology Category

• Machine Learning: Causal Learning
• Computer Vision: Bias, Fairness & Privacy
• Reasoning under Uncertainty: Causality

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Fairness, privacy, and diversity in economic environments

Abstract

Testing for discrimination consists of deriving a profile, known as the comparator, similar to the profile making the discrimination claim, known as the complainant, and comparing the outcomes of these two profiles. An important aspect for establishing discrimination is evidence, often obtained via discrimination testing tools that implement the complainant-comparator pair. In this work, we revisit the role of the comparator in discrimination testing. We argue for the causal modeling nature of deriving the comparator, and introduce a two-kinds classification for the comparator: the ceteris paribus (CP), and mutatis mutandis (MM) comparators. The CP comparator is the standard one among discrimination testing, representing an idealized comparison as it aims for having a complainant-comparator pair that only differs on membership to the protected attribute. As an alternative to it, we define the MM comparator, which requires that the comparator represents what would have been of the complainant without the effects of the protected attribute on the non-protected attributes. The complainant-comparator pair, in that case, may also be dissimilar in terms of all attributes. We illustrate these two comparators and their impact on discrimination testing using a real illustrative example. Importantly, we position generative models and, overall, machine learning methods as useful tools for constructing the MM comparator and, in turn, achieving more complex and realistic comparisons when testing for discrimination.

Problem

• Revisiting the comparator's role in discrimination testing methodology
• Introducing two comparator types: ceteris paribus versus mutatis mutandis
• Proposing mutatis mutandis comparator for complex discrimination analysis

Innovation

• Introduces mutatis mutandis comparator for discrimination testing
• Classifies comparators into ceteris paribus and mutatis mutandis types
• Proposes ML methods for implementing complex comparator adjustments