AgentSys: Secure and Dynamic LLM Agents Through Explicit Hierarchical Memory Management

2026-02-07

AI Summary

This work addresses indirect prompt injection attacks, which exploit the indiscriminate context accumulation mechanism of conventional LLM agents by injecting malicious instructions through external content, leading to persistent harm and degraded decision-making. To mitigate this vulnerability, the authors propose a hierarchical agent architecture inspired by operating system memory isolation: a primary agent invokes worker sub-agents within isolated contexts and permits only structured return values that pass schema validation to cross context boundaries. This approach introduces explicit hierarchical memory management into LLM agents for the first time, effectively severing injection pathways at their source while streamlining agent memory. Experimental results demonstrate that the method reduces attack success rates to 0.78% and 4.25% on the AgentDojo and ASB benchmarks, respectively, while slightly improving performance on benign tasks and maintaining robustness against adaptive attacks across multiple base models.

Technology Category

• Cognitive Modeling & Cognitive Systems: Agent Architectures
• Multiagent Systems: Adversarial Agents
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Search and Retrieval-Augmented AI: Agentic search
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Architectures and workflows that use LLMs for crowd work

Abstract

Indirect prompt injection threatens LLM agents by embedding malicious instructions in external content, enabling unauthorized actions and data theft. LLM agents maintain working memory through their context window, which stores interaction history for decision-making. Conventional agents indiscriminately accumulate all tool outputs and reasoning traces in this memory, creating two critical vulnerabilities: (1) injected instructions persist throughout the workflow, granting attackers multiple opportunities to manipulate behavior, and (2) verbose, non-essential content degrades decision-making capabilities. Existing defenses treat bloated memory as given and focus on remaining resilient, rather than reducing unnecessary accumulation to prevent the attack. We present AgentSys, a framework that defends against indirect prompt injection through explicit memory management. Inspired by process memory isolation in operating systems, AgentSys organizes agents hierarchically: a main agent spawns worker agents for tool calls, each running in an isolated context and able to spawn nested workers for subtasks. External data and subtask traces never enter the main agent's memory; only schema-validated return values can cross boundaries through deterministic JSON parsing. Ablations show isolation alone cuts attack success to 2.19%, and adding a validator/sanitizer further improves defense with event-triggered checks whose overhead scales with operations rather than context length. On AgentDojo and ASB, AgentSys achieves 0.78% and 4.25% attack success while slightly improving benign utility over undefended baselines. It remains robust to adaptive attackers and across multiple foundation models, showing that explicit memory management enables secure, dynamic LLM agent architectures. Our code is available at: https://github.com/ruoyaow/agentsys-memory.

Problem

• indirect prompt injection
• LLM agents
• working memory
• security vulnerability
• context window

Innovation

• hierarchical memory management
• indirect prompt injection defense
• context isolation
• schema-validated communication
• secure LLM agents