Breaking and Fixing Defenses Against Control-Flow Hijacking in Multi-Agent Systems

2025-10-20

AI Summary

Multi-agent systems are vulnerable to control-flow hijacking attacks. Existing semantic-alignment–based defenses (e.g., LlamaFirewall) lack execution-context awareness and rely on fragile “alignment” definitions, rendering them easily bypassable—exposing a fundamental trade-off between security and functionality. This paper proposes ControlValve, a defense framework integrating control-flow integrity (CFI) with the principle of least privilege. ControlValve models permitted control-flow graphs, generates context-sensitive rules via zero-shot prompting, enforces policies through a context-aware policy engine, and monitors execution in real time to enable fine-grained invocation-path control. Experimental evaluation demonstrates that ControlValve effectively mitigates diverse novel hijacking attacks, significantly outperforming state-of-the-art approaches while incurring minimal runtime overhead. Its design ensures practical deployability in real-world multi-agent environments.

Technology Category

• Multiagent Systems: Adversarial Agents
• Computer Vision: Adversarial Attacks & Robustness
• Natural Language Processing: Safety and Robustness

Application Category

• Search and Retrieval-Augmented AI: Agentic search
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Control-flow hijacking attacks manipulate orchestration mechanisms in multi-agent systems into performing unsafe actions that compromise the system and exfiltrate sensitive information. Recently proposed defenses, such as LlamaFirewall, rely on alignment checks of inter-agent communications to ensure that all agent invocations are "related to" and "likely to further" the original objective. We start by demonstrating control-flow hijacking attacks that evade these defenses even if alignment checks are performed by advanced LLMs. We argue that the safety and functionality objectives of multi-agent systems fundamentally conflict with each other. This conflict is exacerbated by the brittle definitions of "alignment" and the checkers' incomplete visibility into the execution context. We then propose, implement, and evaluate ControlValve, a new defense inspired by the principles of control-flow integrity and least privilege. ControlValve (1) generates permitted control-flow graphs for multi-agent systems, and (2) enforces that all executions comply with these graphs, along with contextual rules (generated in a zero-shot manner) for each agent invocation.

Problem

• Evading alignment checks in multi-agent control-flow hijacking
• Resolving safety-functionality conflict in multi-agent systems
• Enforcing control-flow integrity with least privilege principles

Innovation

• Generates permitted control-flow graphs for multi-agent systems
• Enforces execution compliance with generated control-flow graphs
• Applies zero-shot contextual rules for each agent invocation