When Agents Go Rogue: Activation-Based Detection of Malicious Behaviors in Multi-Agent Systems

2026-07-07

AI Summary

This work addresses the challenge of detecting semantically stealthy malicious behaviors in large language model (LLM)-driven multi-agent systems operating under asynchronous conditions without explicit interaction graphs. The paper proposes the first synchronization-agnostic, robust detection and repair framework grounded in the internal activation space of agents. By analyzing agent activation states, the method identifies adversarial compromises without relying on interaction graphs or synchronization assumptions, and subsequently guides compromised agents toward uninterrupted functional recovery. Experimental results demonstrate that the approach achieves F1 scores of 0.94 and 0.93 in synchronous and asynchronous settings, respectively, significantly outperforming graph-based baselines. Moreover, it exhibits strong generalization across diverse open-source LLMs, varying attack intensities, and system scales.

Abstract

While enabling effective collaboration on complex tasks, LLM-based Multi-Agent Systems (MAS) face critical security challenges due to vulnerabilities at the agent and interaction levels. Most existing MAS security defenses are built upon two core assumptions: semantically-explicit malicious attacks and explicit graph-based modeling of the MAS topology and agent-level interactions. In practice, real-world attacks are becoming more semantically stealthy, while MAS execution is typically asynchronous without the temporal alignment assumed by graph-based propagation models. To address these limitations, we propose AcMAS, an activation-based framework for malicious-behavior detection in MAS. By analyzing internal reasoning states in the activation space of local agents, AcMAS detects even stealthy attacks in a synchronization-robust fashion, without relying on explicit interaction graphs. Moreover, our activation analysis provides critical signals to guide AcMAS in restoring the functionality of compromised agents, rather than the disruptive agent isolation commonly used by the state-of-the-art methods. Comprehensive evaluation demonstrates that AcMAS significantly outperforms graph-based baselines against stealthy attacks, by +0.22 F1 in synchronous settings (0.94 vs. 0.72) and by +0.55 F1 in asynchronous settings (0.93 vs. 0.38), with generalization across diverse open-source LLM backbones, attack intensity, and MAS scale.

Problem

• Multi-Agent Systems
• Malicious Behavior Detection
• Stealthy Attacks
• Asynchronous Execution
• Security Vulnerabilities

Innovation

• activation-based detection
• multi-agent systems
• stealthy attacks
• asynchronous execution
• agent recovery