From Plan to Action: How Well Do Agents Follow the Plan?

2026-04-13

AI Summary

It remains unclear whether current programming agents genuinely adhere to prescribed plans or achieve success through data contamination rather than sound reasoning. This work presents the first large-scale, systematic analysis of plan-following behavior in code-generating agents, leveraging the SWE-agent framework to evaluate four large language models across eight plan variants and 16,991 execution trajectories on the SWE-bench Verified and Pro benchmarks. The study finds that high-quality canonical plans substantially improve problem-solving rates, periodic reminders effectively mitigate plan deviation, poorly designed plans can underperform even a no-plan baseline, and prematurely introducing mismatched additional phases degrades performance. These results highlight the critical influence of plan quality, reminder mechanisms, and internal model strategies on task execution outcomes.

Technology Category

• Planning, Routing, and Scheduling: Planning with Language Models
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Multiagent Systems: Multiagent Planning

Application Category

• Search and Retrieval-Augmented AI: Agentic search
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Agents aspire to eliminate the need for task-specific prompt crafting through autonomous reason-act-observe loops. Still, they are commonly instructed to follow a task-specific plan for guidance, e.g., to resolve software issues following phases for navigation, reproduction, patch, and validation. Unfortunately, it is unknown to what extent agents actually follow such instructed plans. Without such an analysis, determining the extent agents comply with a given plan, it is impossible to assess whether a solution was reached through correct strategic reasoning or through other means, e.g., data contamination or overfitting to a benchmark. This paper presents the first extensive, systematic analysis of plan compliance in programming agents, examining 16,991 trajectories from SWE-agent across four LLMs on SWE-bench Verified and SWE-bench Pro under eight plan variations. Without an explicit plan, agents fall back on workflows internalized during training, which are often incomplete, overfit, or inconsistently applied. Providing the standard plan improves issue resolution, and we observe that periodic plan reminders can mitigate plan violations and improve task success. A subpar plan hurts performance even more than no plan at all. Surprisingly, augmenting a plan with additional task-relevant phases in the early stage can degrade performance, particularly when these phases do not align with the model's internal problem-solving strategy. These findings highlight a research gap: fine-tuning paradigms that teach models to follow instructed plans, rather than encoding task-specific plans in them. This requires teaching models to reason and act adaptively, rather than memorizing workflows.

Problem

• plan compliance
• autonomous agents
• task-specific plans
• programming agents
• strategic reasoning

Innovation

• plan compliance
• programming agents
• autonomous reasoning
• instruction following
• workflow generalization