AI Harmonics: a human-centric and harms severity-adaptive AI risk assessment framework

2025-09-12

AI Summary

Current AI risk assessments overemphasize internal compliance, neglecting diverse stakeholder perspectives and real-world societal impacts, thereby failing to comprehensively identify socially embedded harms. Method: We propose a human-centered, severity-adaptive risk assessment paradigm grounded in empirical AI incident data, establishing a dynamic framework that integrates stakeholder perception with statistical modeling. Contribution/Results: We innovatively introduce the ordinal AI Harm metric (AIH), avoiding reliance on precise numerical estimates while enabling comparable quantification and distributional analysis across multidimensional risks—including political, physical, and psychosocial harms. Experiments reveal highest concentration of political and bodily harms, demanding urgent intervention; the framework robustly detects harm inequality across contexts, supporting targeted, evidence-based risk mitigation strategies.

Technology Category

• Philosophy and Ethics of AI: Safety, Robustness & Trustworthiness
• Humans and AI: Learning Human Values and Preferences
• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

The absolute dominance of Artificial Intelligence (AI) introduces unprecedented societal harms and risks. Existing AI risk assessment models focus on internal compliance, often neglecting diverse stakeholder perspectives and real-world consequences. We propose a paradigm shift to a human-centric, harm-severity adaptive approach grounded in empirical incident data. We present AI Harmonics, which includes a novel AI harm assessment metric (AIH) that leverages ordinal severity data to capture relative impact without requiring precise numerical estimates. AI Harmonics combines a robust, generalized methodology with a data-driven, stakeholder-aware framework for exploring and prioritizing AI harms. Experiments on annotated incident data confirm that political and physical harms exhibit the highest concentration and thus warrant urgent mitigation: political harms erode public trust, while physical harms pose serious, even life-threatening risks, underscoring the real-world relevance of our approach. Finally, we demonstrate that AI Harmonics consistently identifies uneven harm distributions, enabling policymakers and organizations to target their mitigation efforts effectively.

Problem

• Addressing AI societal harms through human-centric risk assessment
• Prioritizing AI harms by severity using empirical incident data
• Identifying uneven harm distributions for targeted mitigation efforts

Innovation

• Human-centric harm-severity adaptive framework
• Ordinal severity data without numerical estimates
• Stakeholder-aware data-driven harm prioritization