Echoes of AI Harms: A Human-LLM Synergistic Framework for Bias-Driven Harm Anticipation

2025-11-27

AI Summary

AI systems are increasingly deployed in high-stakes decision-making domains, yet embedded biases throughout their lifecycle pose significant socio-technical risks; existing frameworks typically treat bias and harm in isolation, lacking systematic causal mapping between bias types and concrete harms, while technical interventions remain reactive and insufficiently preventive. Method: We propose the ECHO framework, the first to establish structured, causal links between bias categories and domain-specific harms through stakeholder analysis, context-rich vignettes, and human–large language model collaborative annotation grounded in an ethical matrix—enabling bias–harm pathway identification at design time. Contribution/Results: Validated in high-risk domains—clinical diagnosis and hiring—the framework uncovers domain-specific bias–harm patterns, substantially enhancing pre-deployment risk identification and governance capabilities for AI systems.

Technology Category

• Philosophy and Ethics of AI: Bias, Fairness & Equity
• Machine Learning: Ethics, Bias, and Fairness
• Natural Language Processing: Ethics — Bias, Fairness, Transparency & Privacy

Application Category

• Economics, Online Markets and Human Computation: Fairness and ethical considerations in crowd work and in human-in-the-loop AI systems
• Social Networks and Social Media: Fairness and bias in social network and social media analysis
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

The growing influence of Artificial Intelligence (AI) systems on decision-making in critical domains has exposed their potential to cause significant harms, often rooted in biases embedded across the AI lifecycle. While existing frameworks and taxonomies document bias or harms in isolation, they rarely establish systematic links between specific bias types and the harms they cause, particularly within real-world sociotechnical contexts. Technical fixes proposed to address AI biases are ill-equipped to address them and are typically applied after a system has been developed or deployed, offering limited preventive value. We propose ECHO, a novel framework for proactive AI harm anticipation through the systematic mapping of AI bias types to harm outcomes across diverse stakeholder and domain contexts. ECHO follows a modular workflow encompassing stakeholder identification, vignette-based presentation of biased AI systems, and dual (human-LLM) harm annotation, integrated within ethical matrices for structured interpretation. This human-centered approach enables early-stage detection of bias-to-harm pathways, guiding AI design and governance decisions from the outset. We validate ECHO in two high-stakes domains (disease diagnosis and hiring), revealing domain-specific, bias-to-harm patterns and demonstrating ECHO's potential to support anticipatory governance of AI systems

Problem

• Systematically links AI bias types to harm outcomes
• Enables early-stage detection of bias-to-harm pathways
• Supports anticipatory governance in high-stakes domains

Innovation

• Proactive AI harm anticipation framework
• Human-LLM synergistic bias annotation
• Modular workflow for early-stage detection