"I think this is fair'': Uncovering the Complexities of Stakeholder Decision-Making in AI Fairness Assessment

2025-09-22

AI Summary

AI fairness assessment has long been dominated by technical experts, marginalizing the perspectives of individuals directly affected by algorithmic decisions. Method: This study employs qualitative methods—including semi-structured interviews and scenario-based simulations—to investigate how non-expert stakeholders autonomously evaluate fairness in credit scoring contexts. Contribution/Results: Findings reveal that non-experts extend fairness considerations beyond legally protected attributes to include socioeconomic context; co-design context-embedded, customized fairness metrics; and establish stricter, differentiated, or even individualized fairness thresholds. These judgments demonstrate greater nuance and contextual sensitivity than those produced by conventional expert-driven approaches. The study thus provides empirical grounding and methodological insights for developing more inclusive, participatory AI governance frameworks—highlighting the value of integrating lived experience into fairness evaluation processes.

Technology Category

• Philosophy and Ethics of AI: Bias, Fairness & Equity
• Machine Learning: Ethics, Bias, and Fairness
• Game Theory and Economic Paradigms: Fair Division

Application Category

• Economics, Online Markets and Human Computation: Fairness and ethical considerations in crowd work and in human-in-the-loop AI systems
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Social Networks and Social Media: Fairness and bias in social network and social media analysis

Abstract

Assessing fairness in artificial intelligence (AI) typically involves AI experts who select protected features, fairness metrics, and set fairness thresholds. However, little is known about how stakeholders, particularly those affected by AI outcomes but lacking AI expertise, assess fairness. To address this gap, we conducted a qualitative study with 30 stakeholders without AI expertise, representing potential decision subjects in a credit rating scenario, to examine how they assess fairness when placed in the role of deciding on features with priority, metrics, and thresholds. We reveal that stakeholders' fairness decisions are more complex than typical AI expert practices: they considered features far beyond legally protected features, tailored metrics for specific contexts, set diverse yet stricter fairness thresholds, and even preferred designing customized fairness. Our results extend the understanding of how stakeholders can meaningfully contribute to AI fairness governance and mitigation, underscoring the importance of incorporating stakeholders' nuanced fairness judgments.

Problem

• Examining how non-expert stakeholders assess AI fairness in decision-making scenarios
• Revealing stakeholders consider broader features and stricter thresholds than AI experts
• Addressing the gap in understanding stakeholder contributions to AI fairness governance

Innovation

• Studied non-expert stakeholder fairness assessments
• Revealed stakeholder decisions surpass protected features
• Showed stakeholders prefer customized fairness designs