When Fairness Isn't Statistical: The Limits of Machine Learning in Evaluating Legal Reasoning

2025-06-04

AI Summary

This study examines the boundaries of machine learning for fairness assessment in high-discretion legal domains, using 59,000 Canadian refugee determinations as a case study. Method: It systematically compares three methodological paradigms—feature-based statistical analysis, BERT-powered semantic clustering, and predictive modeling—to evaluate their capacity to capture legally meaningful fairness. Contribution/Results: The study finds a fundamental tension between statistical fairness metrics and juridical fairness; predictive models rely heavily on procedural and contextual features while neglecting substantive legal reasoning; and semantic clustering fails to distinguish critical argumentative structures. Crucially, it provides the first empirical demonstration that purely data-driven approaches are inadequate for legal fairness evaluation—which inherently requires balancing normative values, institutional constraints, and doctrinal reasoning. The work argues that rigorous fairness modeling in law must explicitly integrate doctrinal logic and institutional context. These findings establish a methodological red line for AI deployment in judicial settings.

Technology Category

• Machine Learning: Ethics, Bias, and Fairness
• Philosophy and Ethics of AI: Bias, Fairness & Equity
• Game Theory and Economic Paradigms: Fair Division

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Social Networks and Social Media: Fairness and bias in social network and social media analysis
• Economics, Online Markets and Human Computation: Fairness and ethical considerations in crowd work and in human-in-the-loop AI systems

Abstract

Legal decisions are increasingly evaluated for fairness, consistency, and bias using machine learning (ML) techniques. In high-stakes domains like refugee adjudication, such methods are often applied to detect disparities in outcomes. Yet it remains unclear whether statistical methods can meaningfully assess fairness in legal contexts shaped by discretion, normative complexity, and limited ground truth. In this paper, we empirically evaluate three common ML approaches (feature-based analysis, semantic clustering, and predictive modeling) on a large, real-world dataset of 59,000+ Canadian refugee decisions (AsyLex). Our experiments show that these methods produce divergent and sometimes contradictory signals, that predictive modeling often depends on contextual and procedural features rather than legal features, and that semantic clustering fails to capture substantive legal reasoning. We show limitations of statistical fairness evaluation, challenge the assumption that statistical regularity equates to fairness, and argue that current computational approaches fall short of evaluating fairness in legally discretionary domains. We argue that evaluating fairness in law requires methods grounded not only in data, but in legal reasoning and institutional context.

Problem

• Assessing fairness in legal decisions using machine learning techniques
• Evaluating divergent signals from ML methods in refugee adjudication
• Challenging statistical fairness assumptions in legally discretionary domains

Innovation

• Evaluates ML fairness in legal decisions
• Tests feature-based and semantic methods
• Links legal reasoning to fairness evaluation