🤖 AI Summary
To address fidelity degradation, attention dilution, and high inference cost in LLM agents caused by context explosion during multi-step planning, this paper proposes PAACE, a plan-aware automated context engineering framework. Methodologically, PAACE introduces explicit planning-structure modeling into context compression for the first time, jointly optimizing task relevance, instruction alignment, and function-call coherence, while introducing PAACE-Syn—a step-level supervised synthetic dataset. Based on this, we develop PAACE-FT, a distillable lightweight compressor. Experiments on AppWorld, OfficeBench, and multi-hop QA benchmarks demonstrate significant improvements in accuracy and F1 score, alongside reductions in peak token count, cumulative dependency, and reasoning steps. Notably, PAACE-FT achieves 97% of teacher-model performance at only one-tenth the inference cost.
📝 Abstract
Large Language Model (LLM) agents are increasingly deployed in complex, multi-step workflows involving planning, tool use, reflection, and interaction with external knowledge systems. These workflows generate rapidly expanding contexts that must be curated, transformed, and compressed to maintain fidelity, avoid attention dilution, and reduce inference cost. Prior work on summarization and query-aware compression largely ignores the multi-step, plan-aware nature of agentic reasoning. In this work, we introduce PAACE (Plan-Aware Automated Context Engineering), a unified framework for optimizing the evolving state of LLM agents through next-k-task relevance modeling, plan-structure analysis, instruction co-refinement, and function-preserving compression. PAACE comprises (1) PAACE-Syn, a large-scale generator of synthetic agent workflows annotated with stepwise compression supervision, and (2) PAACE-FT, a family of distilled, plan-aware compressors trained from successful teacher demonstrations. Experiments on long-horizon benchmarks (AppWorld, OfficeBench, and 8-Objective QA) demonstrate that PAACE consistently improves agent correctness while substantially reducing context load. On AppWorld, PAACE achieves higher accuracy than all baselines while lowering peak context and cumulative dependency. On OfficeBench and multi-hop QA, PAACE improves both accuracy and F1, achieving fewer steps, lower peak tokens, and reduced attention dependency. Distilled PAACE-FT retains 97 percent of the teacher's performance while reducing inference cost by over an order of magnitude, enabling practical deployment of plan-aware compression with compact models.