🤖 AI Summary
Interactive AI agents introduce emergent systemic risks at the system level—particularly unpredictable failures arising from multi-agent coordination. Method: We propose a scenario-driven risk identification paradigm, construct representative risk-evolution case studies spanning smart grids and social welfare domains, and introduce Agentology—a novel graphical modeling language for capturing complex agent interactions. We systematically identify and categorize emergent behaviors that trigger systemic risks, establishing the first hierarchical risk taxonomy specifically designed for interactive AI systems. Contribution/Results: Our work yields a cross-domain, transferable risk analysis framework and pioneers the research direction of AI system-level safety. It provides both theoretical foundations and methodological tools for designing and governing high-reliability multi-agent systems, advancing rigorous, scalable approaches to AI safety beyond component-level assurance.
📝 Abstract
In this study, we investigate system-level emergent risks of interacting AI agents. The core contribution of this work is an exploratory scenario-based identification of these risks as well as their categorization. We consider a multitude of systemic risk examples from existing literature and develop two scenarios demonstrating emergent risk patterns in domains of smart grid and social welfare. We provide a taxonomy of identified risks that categorizes them in different groups. In addition, we make two other important contributions: first, we identify what emergent behavior types produce systemic risks, and second, we develop a graphical language "Agentology" for visualization of interacting AI systems. Our study opens a new research direction for system-level risks of interacting AI, and is the first to closely investigate them.