ECHO: Prune to act, trace to learn with selective turn memory in agentic RL

2026-06-30

AI Summary

This work addresses the limitations of long-horizon language agents, which struggle to retain fine-grained historical evidence due to finite context windows and suffer from non-traceable credit assignment in reinforcement learning when using existing history compression methods. To jointly tackle history collapse and untraceable policy learning, the authors propose the ECHO framework, which generates compact memory records at each interaction step and employs a source-indexing mechanism to selectively reconstruct policy-relevant contexts. This enables efficient memory utilization and traceable credit assignment. Evaluated on BrowseComp-Plus, ECHO achieves a 43.4% accuracy—substantially outperforming GRPO (28.9%) and SUPO (36.1%)—while reducing both interaction steps and trajectory size, and enhancing zero-shot generalization across diverse tasks such as multi-hop question answering and code generation.

Abstract

Long-horizon language agents must repeatedly interact with tools, accumulate evidence, and make decisions under bounded context windows. Existing context-management methods make such rollouts feasible by truncating distant history, folding past turns into summaries, or selecting compact memory states. However, these breakthroughs introduce two coupled limitations. First, as the number of turns grows, historical observations are progressively removed or collapsed into compressed states, making it harder for the policy to reuse fine-grained evidence. Second, once the original turns are no longer source-addressable, outcome-based RL loses an explicit path for aligning policy updates with the evidence that supported a successful final answer. To this end, we propose ECHO, a selective turn-memory framework that jointly addresses history collapse and traceable learning through source-indexed reconstruction. Specifically, ECHO compresses each completed environment turn into a compact memory record, reconstructs bounded policy contexts by selecting from these records, and reuses the selected source indices to route positive outcome credit to the evidence and selection actions that support successful answers. On BrowseComp-Plus, ECHO reaches 43.4% held-out accuracy, outperforming GRPO (28.9%) and the rolling-summary baseline SUPO (36.1%), while using fewer turns and lower trajectory volume than SUPO (Figure 1). Additionally, the trained policy improves zero-shot generalization across multi-objective QA, code generation, and deep information-seeking benchmarks on both dense and MoE backbones.

Problem

• long-horizon language agents
• context management
• history collapse
• traceable learning
• evidence reuse

Innovation

• selective turn memory
• source-indexed reconstruction
• traceable reinforcement learning
• long-horizon language agents
• evidence-aware policy learning