Generative AI and Perceptual Harms: Who's Suspected of using LLMs?

2024-10-01

arXiv.org

AI Summary

This study introduces and empirically tests the concept of “perceived harm”—negative evaluations and discrimination arising from others’ suspicion that an individual uses large language models (LLMs). Method: Three online experiments manipulated AI-use cues in fictitious freelance writer profiles and assessed their impact on writing quality ratings and hiring intentions using Likert-scale measures and statistical modeling (logistic regression, mediation analysis). Contribution/Results: Suspicion of AI use significantly reduced perceived writing quality and hiring likelihood; this effect was robust across demographic groups and amplified for certain marginalized populations. The study provides the first systematic empirical evidence of generative AI–induced stigma and its structural inequity consequences, advancing AI ethics and fairness research by proposing a novel theoretical framework grounded in rigorous experimental evidence.

Technology Category

• Philosophy and Ethics of AI: Bias, Fairness & Equity
• Natural Language Processing: Ethics — Bias, Fairness, Transparency & Privacy
• Humans and AI: AI for Accessibility

Application Category

• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Large language models (LLMs) are increasingly integrated into a variety of writing tasks. While these tools can help people by generating ideas or producing higher quality work, like many other AI tools they may risk causing a variety of harms, disproportionately burdening historically marginalized groups. In this work, we introduce and evaluate perceptual harm, a term for the harm caused to users when others perceive or suspect them of using AI. We examined perceptual harms in three online experiments, each of which entailed human participants evaluating the profiles for fictional freelance writers. We asked participants whether they suspected the freelancers of using AI, the quality of their writing, and whether they should be hired. We found some support for perceptual harms against for certain demographic groups, but that perceptions of AI use negatively impacted writing evaluations and hiring outcomes across the board.

Problem

• Examines perceptual harms from suspected AI use in writing.
• Investigates impact of AI suspicion on writing evaluations and hiring.
• Assesses disproportionate effects on historically marginalized groups.

Innovation

• Evaluated perceptual harms from suspected AI use
• Conducted online experiments with fictional writer profiles
• Assessed impact on writing evaluations and hiring outcomes