Architecting Resilient LLM Agents: A Guide to Secure Plan-then-Execute Implementations

2025-09-10

AI Summary

To address the vulnerabilities of LLM-based agents in multi-step tasks—including weak security guarantees, low predictability, and susceptibility to indirect prompt injection attacks—this paper proposes a high-resilience agent architecture ensuring control-flow integrity. The core methodology adopts a strict “plan-then-execute” paradigm, decoupling strategic planning from tactical execution, and integrates declarative tool scoping, DAG-driven state graphs, Docker-based sandboxing, and human-in-the-loop verification. The architecture enables dynamic replanning and secure parallel execution, implementing defense-in-depth. A prototype implemented atop LangGraph, CrewAI, and AutoGen demonstrates substantial improvements over reactive paradigms (e.g., ReAct) in reasoning quality, security robustness, and production-grade reliability—while maintaining cost efficiency. This work delivers a reusable architectural blueprint and practical engineering framework for trustworthy LLM agents.

Technology Category

• Cognitive Modeling & Cognitive Systems: Agent Architectures
• Multiagent Systems: Multiagent Planning
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Search and Retrieval-Augmented AI: Agentic search
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

As Large Language Model (LLM) agents become increasingly capable of automating complex, multi-step tasks, the need for robust, secure, and predictable architectural patterns is paramount. This paper provides a comprehensive guide to the ``Plan-then-Execute'' (P-t-E) pattern, an agentic design that separates strategic planning from tactical execution. We explore the foundational principles of P-t-E, detailing its core components - the Planner and the Executor - and its architectural advantages in predictability, cost-efficiency, and reasoning quality over reactive patterns like ReAct (Reason + Act). A central focus is placed on the security implications of this design, particularly its inherent resilience to indirect prompt injection attacks by establishing control-flow integrity. We argue that while P-t-E provides a strong foundation, a defense-in-depth strategy is necessary, and we detail essential complementary controls such as the Principle of Least Privilege, task-scoped tool access, and sandboxed code execution. To make these principles actionable, this guide provides detailed implementation blueprints and working code references for three leading agentic frameworks: LangChain (via LangGraph), CrewAI, and AutoGen. Each framework's approach to implementing the P-t-E pattern is analyzed, highlighting unique features like LangGraph's stateful graphs for re-planning, CrewAI's declarative tool scoping for security, and AutoGen's built-in Docker sandboxing. Finally, we discuss advanced patterns, including dynamic re-planning loops, parallel execution with Directed Acyclic Graphs (DAGs), and the critical role of Human-in-the-Loop (HITL) verification, to offer a complete strategic blueprint for architects, developers, and security engineers aiming to build production-grade, resilient, and trustworthy LLM agents.

Problem

• Securing LLM agents against indirect prompt injection attacks
• Designing resilient Plan-then-Execute architecture for agents
• Implementing defense strategies for production-grade LLM agents

Innovation

• Plan-then-Execute pattern separates planning from execution
• Implements control-flow integrity against prompt injection attacks
• Uses sandboxed code execution and least privilege controls