ASCon: A Direction-Aware Reciprocal Agent--Step Contextualization Model for Failure Attribution in Multi-Agent Systems

2026-08-11

AI Summary

This work addresses the problem of attribution in multi-agent systems—specifically, identifying faulty agents, erroneous steps, failure modes, and their underlying causes and timing. To this end, the authors propose a unified representation model that aggregates shared diagnostic evidence from task trajectories, including constraints, roles, behavioral histories, and interactions. The model introduces a direction-aware, mutually recursive agent-step contextualization mechanism to jointly capture dependencies among multiple attribution objectives, enabling knowledge sharing and transfer. It integrates direction-aware graph attention, masked step-to-agent attention, and agent-conditioned step contextualization, complemented by a lightweight task head. Experiments demonstrate substantial improvements: micro-accuracy for faulty agent and step detection increases by over 5.83% and 10.63%, respectively, while Macro-F1 for failure mode detection improves by more than 14.73%. The approach also significantly enhances large language models’ attribution capabilities in out-of-domain scenarios.

Abstract

Failure attribution in LLM-based multi-agent systems (MAS) aims to answer who caused failures, when they occurred, and why by identifying responsible targets including faulty agents, erroneous steps, and failure modes. Existing methods have primarily focused on developing dedicated models for specific attribution targets, with limited attention to the evidential dependencies among them. Despite these attribution targets are different, they rely on common diagnostic evidence from MAS trajectories, including task constraints, agent roles, behavioral histories and inter-agent interactions. This commonality motivates us to develop a unified representation model that aggregates the trajectory evidence into individual agent and step representations, which can subsequently be adapted to different attribution targets. Accordingly, we propose ASCon, a direction-aware reciprocal \textbf{A}gent--\textbf{S}tep \textbf{Con}textualization model for multiple failure attribution targets. ASCon introduces direction-aware graph attention to model execution context, masked step-to-agent attention to construct behavior-aware agent representations, and agent-conditioned step contextualization to incorporate agent context back into step representations. The resulting contextualized representations enable different attribution targets through lightweight target-specific heads. Experiments show that ASCon can improve faulty-agent detection by 5.83\%+ in micro-accuracy, faulty-step detection by 10.63\%+ in micro-accuracy, and failure-mode detection by 14.73\%+ in Macro-F1. Meanwhile, it can also substantially enhance the LLM-based methods' attribution capabilities in out-of-domain scenarios.

Problem

• failure attribution
• multi-agent systems
• faulty agents
• faulty steps
• failure modes

Innovation

• failure attribution
• multi-agent systems
• direction-aware graph attention
• agent-step contextualization
• unified representation