Interpreting Emergent Extreme Events in Multi-Agent Systems

2026-01-28

AI Summary

This work addresses the safety risks posed by unexplained extreme events emerging from large language model–driven interactions in multi-agent systems. To tackle this challenge, we propose the first explainable attribution framework specifically designed for such emergent phenomena. Our approach innovatively adapts Shapley values to temporal multi-agent behavior analysis, quantifying the contribution of each action across three dimensions—time, agent, and behavior—at every time step. We further introduce a multidimensional aggregation mechanism and tailored metrics to characterize extreme events. Extensive experiments across diverse domains, including economics, finance, and social simulations, demonstrate the framework’s effectiveness and uncover common patterns underlying the emergence of extreme behaviors in multi-agent systems.

Technology Category

• Multiagent Systems: Agent-Based Simulation and Emergent Behavior
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Philosophy and Ethics of AI: Accountability, Interpretability & Explainability

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Search and Retrieval-Augmented AI: Agentic search

Abstract

Large language model-powered multi-agent systems have emerged as powerful tools for simulating complex human-like systems. The interactions within these systems often lead to extreme events whose origins remain obscured by the black box of emergence. Interpreting these events is critical for system safety. This paper proposes the first framework for explaining emergent extreme events in multi-agent systems, aiming to answer three fundamental questions: When does the event originate? Who drives it? And what behaviors contribute to it? Specifically, we adapt the Shapley value to faithfully attribute the occurrence of extreme events to each action taken by agents at different time steps, i.e., assigning an attribution score to the action to measure its influence on the event. We then aggregate the attribution scores along the dimensions of time, agent, and behavior to quantify the risk contribution of each dimension. Finally, we design a set of metrics based on these contribution scores to characterize the features of extreme events. Experiments across diverse multi-agent system scenarios (economic, financial, and social) demonstrate the effectiveness of our framework and provide general insights into the emergence of extreme phenomena.

Problem

• emergent extreme events
• multi-agent systems
• interpretability
• system safety
• black-box emergence

Innovation

• emergent extreme events
• multi-agent systems
• Shapley value attribution
• interpretable AI
• risk contribution analysis