PEEK: Context Map as an Orientation Cache for Long-Context LLM Agents

2026-05-19

AI Summary

This work addresses the limited persistent awareness of structure, content, and historically useful information in long-context large language model agents when repeatedly accessing external context. To overcome this, the authors propose ContextMap—a compact, constant-size caching mechanism that enables reusable, context-guided knowledge caching for the first time. ContextMap comprises three components: a Distiller that extracts transferable knowledge, a Cartographer that generates structured edits, and an Evictor that dynamically maintains the cache based on priority. Under a fixed token budget, the method achieves performance gains of 6.3–34.0% over the strongest baselines on long-context reasoning and information aggregation tasks, reduces iteration counts by 93–145, and lowers computational costs by 1.7–5.8×. In in-context learning tasks, it improves solution accuracy by 6.0–14.0% and scoring accuracy by 7.8–12.1%, with a 1.4× cost reduction.

Abstract

Large language model (LLM) agents increasingly operate over long and recurring external contexts, like document corpora and code repositories. Across invocations, existing approaches preserve either the agent's trajectory, passive access to raw material, or task-level strategies. None of them preserves what we argue is most needed for repeated same-context workloads: reusable orientation knowledge (e.g., what the context contains, how it is organized, and which entities, constants, and schemas have historically been useful) about the recurring context itself. We introduce PEEK, a system that caches and maintains this orientation knowledge as a context map: a small, constant-sized artifact in the agent's prompt that gives it a persistent peek into the external context. The map is maintained by a programmable cache policy with three modules: a Distiller that extracts transferable knowledge from inference-time signals, a Cartographer that translates it into structured edits, and a priority-based Evictor that enforces a fixed token budget. On long-context reasoning and information aggregation, PEEK improves over strong baselines by 6.3-34.0% while using 93-145 fewer iterations and incurring 1.7-5.8x lower cost than the state-of-the-art prompt-learning framework, ACE. On context learning, PEEK improves solving rate and rubric accuracy by 6.0-14.0% and 7.8-12.1%, respectively, at 1.4x lower cost than ACE. These gains generalize across LMs and agent architectures, including OpenAI Codex, a production-grade coding agent. Together, these results show that a context map helps long-context LLM agents interact with recurring external contexts more accurately and efficiently.

Problem

• long-context LLM agents
• recurring external contexts
• orientation knowledge
• context map
• reusable knowledge

Innovation

• context map
• orientation knowledge
• long-context LLM agents
• programmable cache policy
• efficient prompting