Robust Multi-Agent LLMs under Byzantine Faults

2026-05-09

AI Summary

This work addresses the vulnerability of decentralized multi-agent large language model (LLM) systems to Byzantine faults, which can trigger error propagation and performance degradation. To mitigate this, the authors propose Self-Anchored Consensus (SAC), a fully decentralized protocol that achieves Byzantine-resilient consensus without relying on a leader or self-reported confidence scores. SAC leverages iterative message exchange, local filtering of unreliable information, and output refinement mechanisms. The study establishes, for the first time, an (F+1)-robustness graph condition tailored to decentralized LLM-based multi-agent systems, integrating graph-theoretic analysis with LLM reasoning. Experimental results demonstrate that SAC significantly suppresses Byzantine influence across diverse communication topologies and consistently outperforms existing methods in both mathematical and commonsense reasoning tasks.

Abstract

Large language model (LLM) agents increasingly collaborate over peer-to-peer networks to improve their reliability. However, these same interactions can also become a source of vulnerability, as unreliable or Byzantine agents may sway neighboring agents toward incorrect conclusions and degrade overall system performance. Existing methods rely on leader-based coordination or self-reported confidence, both of which are susceptible to adversarial manipulation. We study decentralized LLM multi-agent systems (LLM-MAS) and propose Self-Anchored Consensus (SAC), a fully decentralized iterative filter-and-refine protocol in which agents iteratively exchange responses, locally evaluate and filter unreliable messages, and refine their own outputs. We present $(F{+}1)$-robustness conditions for the communication graph that ensure honest agents preserve and propagate reliable information despite Byzantine influence. Experiments on mathematical and commonsense reasoning benchmarks show that SAC effectively suppresses Byzantine influence and consistently improves performance across diverse communication topologies, whereas prior methods degrade under adversarial conditions.

Problem

• Byzantine faults
• multi-agent LLMs
• decentralized systems
• robustness
• adversarial manipulation

Innovation

• Byzantine fault tolerance
• decentralized consensus
• multi-agent LLMs
• Self-Anchored Consensus
• robustness