Beyond LLM-Based Reasoning: Lightweight GNNs for Agent Failure Attribution

2026-08-19

AI Summary

为解决多智能体系统中错误归因问题，提出一种轻量级图神经网络AFANet，通过语义信号和关系建模，以较低计算成本达到或超越现有基于大语言模型方法的性能。

Abstract

Large language model (LLM)-based multi-agent systems (MAS) often exhibit complex failure modes, which frequently cause agents to produce incorrect outcomes. This motivates the task of Agent Failure Attribution: given a failed multi-agent trajectory, identify the faulty agents and their corresponding error types. Existing approaches predominantly rely on LLMs to perform failure attribution, either through direct prompting, fine-tuning on synthetic data or complex agentic pipelines. While effective, these methods incur substantial computational overhead due to long-context processing, expensive post-training and handcrafted workflows. Moreover, empirical evidence shows that even state-of-the-art models achieve limited accuracy on existing benchmarks, suggesting that scaling model size alone is insufficient. In this work, we revisit this task and question the necessity of such expensive generative solutions. We introduce AFANet, a lightweight graph-based framework that models interaction trajectories through step-level semantic signals and agent-level relationships. We show that with significantly fewer parameters and near-zero inference cost, AFANet (i) matches or outperforms LLM-based baselines, including fine-tuned models on in-domain benchmarks, (ii) maintains robust performance across different GNN architectures and (iii) can be further improved with inexpensive test-time adaptation on the OOD benchmark. Our results suggest that effective agent failure attribution does not require heavy LLM reasoning and a lightweight, structured approach can achieve strong performance.

Problem

• Agent Failure Attribution
• Multi-Agent Systems
• Large Language Models
• Faulty Agents
• Error Types

Innovation

• Lightweight GNNs
• Agent Failure Attribution
• Step-level Semantic Signals
• Low Inference Cost