"Cold, Calculated, and Condescending": How AI Identifies and Explains Ableism Compared to Disabled People

2024-10-04

AI Summary

This study identifies systemic biases in current AI content moderation systems regarding the detection and interpretation of ableist speech. We construct the first disability-focused dataset of 200 social media comments, then employ toxicity classifiers and large language models (LLMs) to generate multi-dimensional toxicity and ableism scores alongside attribution rationales. A mixed-methods investigation—including large-scale annotation by 190 disabled participants and in-depth qualitative analysis—enables the first empirical comparison between AI judgments and disabled individuals’ lived perceptions of harm. Results reveal that AI systems consistently underestimate toxicity, exhibit poor inter-rater agreement on ableism identification, and produce attributions rife with stereotypes, erroneous assumptions, and patronizing language—diverging markedly from disabled users’ experiential realities. Accordingly, we propose a reconfigured AI ethics evaluation framework grounded in intersectional disability justice, shifting content moderation paradigms from technical determinism toward disabled epistemic authority and subjectivity.

Technology Category

• Philosophy and Ethics of AI: Bias, Fairness & Equity
• Humans and AI: AI for Accessibility
• Machine Learning: Ethics, Bias, and Fairness

Application Category

• Social Networks and Social Media: Fairness and bias in social network and social media analysis
• Economics, Online Markets and Human Computation: Fairness and ethical considerations in crowd work and in human-in-the-loop AI systems
• User Modeling, Personalization and Recommendation: Accountability, Transparency, and Ethics for personalization

Abstract

People with disabilities (PwD) regularly encounter ableist hate and microaggressions online. These spaces are generally moderated by machine learning models, but little is known about how effectively AI models identify ableist speech and how well their judgments align with PwD. To investigate this, we curated a first-of-its-kind dataset of 200 social media comments targeted towards PwD, and prompted state-of-the art AI models (i.e., Toxicity Classifiers, LLMs) to score toxicity and ableism for each comment, and explain their reasoning. Then, we recruited 190 participants to similarly rate and explain the harm, and evaluate LLM explanations. Our mixed-methods analysis highlighted a major disconnect: AI underestimated toxicity compared to PwD ratings, while its ableism assessments were sporadic and varied. Although LLMs identified some biases, its explanations were flawed--they lacked nuance, made incorrect assumptions, and appeared judgmental instead of educational. Going forward, we discuss challenges and opportunities in designing moderation systems for ableism, and advocate for the involvement of intersectional disabled perspectives in AI.

Problem

• Artificial Intelligence
• Disability Discrimination
• Inclusive Language

Innovation

• Artificial Intelligence
• Disability Perspectives
• Online Harassment Detection