Designing Incident Reporting Systems for Harms from General-Purpose AI

2025-11-08

AI Summary

In response to escalating safety and rights risks posed by general-purpose artificial intelligence (GPAI), this paper proposes the first systematic reporting framework for GPAI incidents. Drawing on a systematic literature review and cross-case analysis of high-stakes domains—including aviation and healthcare—as well as regulatory practices in the U.S. and EU, the study identifies seven core dimensions: policy objectives, reporting entities, incident typologies, reporting modalities (mandatory vs. voluntary), near-miss inclusion, anonymity safeguards, and legal immunity provisions. It critically examines the trade-offs among safety learning, cross-organizational information sharing, and legal interoperability inherent in each mechanism. The resulting framework offers policymakers and researchers an actionable, theory-informed blueprint for designing GPAI incident reporting infrastructure—addressing a critical gap in GPAI risk governance and advancing the institutional foundations for responsible AI development and deployment.

Technology Category

• Philosophy and Ethics of AI: Safety, Robustness & Trustworthiness
• Natural Language Processing: Safety and Robustness
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Security and Privacy: Security and privacy of machine learning and AI applications
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems

Abstract

We introduce a conceptual framework and provide considerations for the institutional design of AI incident reporting systems, i.e., processes for collecting information about safety- and rights-related events caused by general-purpose AI. As general-purpose AI systems are increasingly adopted, they are causing more real-world harms and displaying the potential to cause significantly more dangerous incidents - events that did or could have caused harm to individuals, property, or the environment. Through a literature review, we develop a framework for understanding the institutional design of AI incident reporting systems, which includes seven dimensions: policy goal, actors submitting and receiving reports, type of incidents reported, level of risk materialization, enforcement of reporting, anonymity of reporters, and post-reporting actions. We then examine nine case studies of incident reporting in safety-critical industries to extract design considerations for AI incident reporting in the United States. We discuss, among other factors, differences in systems operated by regulatory vs. non-regulatory government agencies, near miss reporting, the roles of mandatory reporting thresholds and voluntary reporting channels, how to enable safety learning after reporting, sharing incident information, and clarifying legal frameworks for reporting. Our aim is to inform researchers and policymakers about when particular design choices might be more or less appropriate for AI incident reporting.

Problem

• Developing institutional frameworks for AI incident reporting systems
• Addressing safety and rights harms from general-purpose AI adoption
• Establishing design dimensions for effective incident reporting processes

Innovation

• Developed conceptual framework for AI incident reporting
• Identified seven key dimensions for institutional design
• Analyzed case studies to extract design considerations