When Local Monitors Miss Compositional Harm: Diagnosing Distributed Backdoors in Multi-Agent Systems

2026-07-13

AI Summary

In multi-agent large language model systems, adversaries can distribute malicious behavior across multiple agents such that individual local observations appear benign, while their collective interaction constitutes a backdoor attack—rendering conventional local monitoring ineffective. This work formally defines, for the first time, the notions of local harmlessness and observability boundaries for such “distributed backdoors,” elucidating the fundamental reason why local monitors fail to detect compositional attacks. Through controlled experiments, benchmark evaluations, and end-to-end agent assessments, the study introduces a decoding-view-based detector achieving an average AUROC of 0.874 on unseen encodings, along with a gating mechanism that completely blocks all tested distributed backdoor attacks.

Abstract

As multi-agent, tool-using LLM systems are deployed, a common safety net is a runtime monitor that checks each message, tool call, or step on its own. We show this net has a fundamental hole. A distributed backdoor splits a harmful payload across agents, so every local check passes while the assembled object is the attack. The monitor can be right on every step and still miss the attack. The problem is not splitting itself: split fragments can still leak suspicious tokens or provenance edges. The hard case is \emph{local benignness}. No fragment carries the harm, and what is left looks like ordinary benign traffic. We formalize this as an \emph{observability boundary}: a monitor catches only what its view can tell apart from benign traffic. We prove that once the fragments look benign in the monitored view, no detector on that view can catch them, however strong it is. Across a controlled testbed, an external benchmark, and end-to-end agent runs, local monitors lose the signal exactly as local evidence disappears, and it returns only when the monitor sees the assembled object. A monitor trained only on benign traffic recovers the attack's code structure across held-out encodings (0.874 mean AUROC). A decoded-view gate, given the encoding family, blocks every tested attack. But seeing more is not enough: full-trace monitors and decoders still fail unless they reach the representation where the payload is exposed. Local safety is not global safety when harm is compositional, and the open problem is finding that representation.

Problem

• distributed backdoors
• multi-agent systems
• local benignness
• observability boundary
• compositional harm

Innovation

• distributed backdoors
• local benignness
• observability boundary
• multi-agent systems
• compositional harm