Coding Agents are Effective Long-Context Processors

2026-03-20

AI Summary

This work addresses the significant performance degradation of large language models (LLMs) when processing extremely long contexts, a limitation rooted in their reliance on implicit attention mechanisms. To overcome this, the authors propose a novel paradigm that externalizes long-context processing tasks to coding agents capable of executing explicit operations. Leveraging these agents’ native proficiency with file systems, code, and terminal commands, the approach explicitly structures and manipulates textual content, replacing conventional semantic querying. Built upon state-of-the-art off-the-shelf coding agents, the method is evaluated on corpora containing up to three trillion tokens and demonstrates an average improvement of 17.3% over current best-performing approaches across multiple long-context benchmarks, substantially enhancing both efficiency and scalability.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Multiagent Systems: Modeling other Agents

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Large Language Models (LLMs) have demonstrated remarkable progress in scaling to access massive contexts. However, the access is via the latent and uninterpretable attention mechanisms, and LLMs fail to effective process long context, exhibiting significant performance degradation as context length increases. In this work, we study whether long-context processing can be externalized from latent attention into explicit, executable interactions, by allowing coding agents to organize text in file systems and manipulate it using its native tools. We evaluate off-the-shelf frontier coding agents as the general interface for tasks that require processing long contexts, including long-context reasoning, retrieval-augmented generation, and open-domain question answering with large-scale corpus contains up to three trillion tokens. Across multiple benchmarks, these agents outperform published state-of-the-art by 17.3% on average. We attribute this efficacy to two key factors: native tool proficiency, which enables agents to leverage executable code and terminal commands rather than passive semantic queries, and file system familiarity, which allows them to navigate massive text corpora as directory structures. These findings suggest that delegating long-context processing to coding agents offers an effective alternative to semantic search or context window scaling, opening new directions for long-context processing in LLMs.

Problem

• long-context processing
• Large Language Models
• attention mechanisms
• performance degradation
• context length

Innovation

• coding agents
• long-context processing
• file system navigation
• executable interaction
• tool proficiency