Role and Use of Race in AI/ML Models Related to Health

2025-04-01

AI Summary

The use of race as a variable in health AI/ML models lacks a systematic ethical governance framework across the model lifecycle. Method: This paper introduces the first comprehensive, lifecycle-oriented analytical framework—spanning data collection, modeling, deployment, and impact assessment—integrating qualitative systematic review, multi-stakeholder perspectives, and ethics-technology co-analysis, while foregrounding race’s social construction and clinical context-sensitivity. Contribution/Results: It proposes a structured set of “considerations” as a decision-support toolkit, specifying criteria for determining the necessity of race variables, identifying ethically and technically viable alternatives, and enforcing transparency requirements. The framework addresses a critical gap in guidance on structural fairness within health AI governance and advances consensus among researchers and regulators on equitable practices, establishing an actionable, responsible benchmark for race-variable usage in clinical AI systems.

Technology Category

• Philosophy and Ethics of AI: Bias, Fairness & Equity
• Machine Learning: Ethics, Bias, and Fairness
• Natural Language Processing: Ethics — Bias, Fairness, Transparency & Privacy

Application Category

• Economics, Online Markets and Human Computation: Fairness and ethical considerations in crowd work and in human-in-the-loop AI systems
• User Modeling, Personalization and Recommendation: Accountability, Transparency, and Ethics for personalization
• Responsible Web: Fairness, accountability, transparency and ethics of web technologies

Abstract

The role and use of race within health-related artificial intelligence and machine learning (AI/ML) models has sparked increasing attention and controversy. Despite the complexity and breadth of related issues, a robust and holistic framework to guide stakeholders in their examination and resolution remains lacking. This perspective provides a broad-based, systematic, and cross-cutting landscape analysis of race-related challenges, structured around the AI/ML lifecycle and framed through"points to consider"to support inquiry and decision-making.

Problem

• Examining race's role in health AI/ML models
• Lacking framework for race-related AI/ML challenges
• Providing systematic analysis for decision-making

Innovation

• Holistic framework for race-related AI/ML challenges
• Systematic analysis across AI/ML lifecycle
• Points to consider for decision-making