Explanatory Debiasing: Involving Domain Experts in the Data Generation Process to Mitigate Representation Bias in AI Systems

2024-12-26

arXiv.org

AI Summary

AI systems often exhibit representational bias due to insufficient domain knowledge, leading to poor generalization across underrepresented subpopulations. Method: This paper proposes a human–AI collaborative paradigm for representation debiasing, systematically embedding domain experts throughout the data generation and bias identification pipeline to enable structured injection of expert knowledge at the data level. Diverging from conventional algorithm-level debiasing approaches, we introduce the first generalizable human–AI co-design guideline specifically for representation debiasing and establish a mixed-methods research framework, empirically validated in healthcare. Contribution/Results: Experiments involving 35 clinical experts demonstrate statistically significant reduction in bias affecting underrepresented subgroups (p < 0.01), without compromising overall model accuracy. The study yields a reusable, expert-informed debiasing implementation protocol, offering a novel pathway for trustworthy AI governance through data-centric interventions.

Technology Category

• Humans and AI: Learning Human Values and Preferences
• Philosophy and Ethics of AI: Bias, Fairness & Equity
• Machine Learning: Ethics, Bias, and Fairness

Application Category

• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Representation bias is one of the most common types of biases in artificial intelligence (AI) systems, causing AI models to perform poorly on underrepresented data segments. Although AI practitioners use various methods to reduce representation bias, their effectiveness is often constrained by insufficient domain knowledge in the debiasing process. To address this gap, this paper introduces a set of generic design guidelines for effectively involving domain experts in representation debiasing. We instantiated our proposed guidelines in a healthcare-focused application and evaluated them through a comprehensive mixed-methods user study with 35 healthcare experts. Our findings show that involving domain experts can reduce representation bias without compromising model accuracy. Based on our findings, we also offer recommendations for developers to build robust debiasing systems guided by our generic design guidelines, ensuring more effective inclusion of domain experts in the debiasing process.

Problem

• Mitigates representation bias in AI systems
• Involves domain experts in debiasing process
• Enhances AI model accuracy across data segments

Innovation

• Domain experts enhance debiasing
• Design guidelines for expert involvement
• Healthcare application validates effectiveness