The Quiet Path from Seemingly Minor Design Errors to Workplace AI Incidents

2026-05-20

AI Summary

This study identifies a critical misalignment between the design of workplace AI systems and the actual needs of workers as a primary cause of workplace incidents. Through analysis of 1,524 AI-related accident reports, the research quantifies for the first time that 83% of incidents stem from human-AI misalignment, with 74% attributable to developers’ overemphasis on efficiency and speed. The work innovatively employs a “large language model as an expert” (LLM-as-an-expert) approach, integrated with a twelve-dimensional AI trait framework, to systematically evaluate these discrepancies. Findings reveal that workers favor AI systems that are accurate, insightful, or personalized, whereas deployed systems are predominantly basic and generic. Moreover, the proliferation of generative AI has introduced novel incidents linked to AI systems exhibiting “imagination,” highlighting emergent risks in increasingly autonomous and creative AI applications.

Abstract

Recent human-computer interaction (HCI) research has revealed a widespread misalignment between how developers design workplace artificial intelligence (AI) systems, and what workers actually need from them. Yet, little research has examined the effects of this gap, or how it may cause harm. We analyzed 1,524 reports of incidents in which AI systems were used to perform 171 occupational tasks across 12 industry sectors. Using an Large Language Model (LLM)-as-an-expert approach, we extracted the main traits of the AI systems involved in those incidents using an established framework of twelve traits. We then compared them with the traits that 202 workers highly familiar with those tasks would have preferred. We found that as many as 83\% of workplace incidents stem from worker-AI misalignments. In most cases, workers wanted systems that are precise, insightful, or personal, but instead received systems that are basic, simple, or general. Over the years, fast AI caused a considerable number of incidents, yet these declined, and imaginative AI, with the mass introduction of generative AI, started to cause incidents. We also compared the traits causing the incidents with the traits that 197 developers building AI systems for those tasks would have preferred. If the traits causing the incidents were the same as those designed by developers, then developers may be responsible for those incidents. We found that 74\% of task misalignments could be attributed to developers who tended to overfocus on efficiency and speed, especially for systems performing tasks in people-facing occupations such as those in the human resources sector. Our results call for design interventions that better align AI development with workers' needs, as without such corrections, workplace AI incidents are likely to persist, causing the invisible erosion of worker agency and organizational productivity.

Problem

• workplace AI incidents
• worker-AI misalignment
• AI design errors
• occupational tasks
• developer-worker gap

Innovation

• worker-AI misalignment
• LLM-as-an-expert
• workplace AI incidents
• design preferences
• generative AI