ReContext: Recursive Evidence Replay as LLM Harness for Long-Context Reasoning

2026-07-02

AI Summary

Although large language models support long contexts, they often struggle to effectively leverage relevant evidence within the input for reasoning. This work proposes a training-free, recursive evidence replay method that constructs a query-specific evidence pool based on attention mechanisms and recursively replays critical evidence while preserving the full original context to enhance reasoning capabilities. It is the first to introduce recursive evidence replay into long-context reasoning, decoupling evidence organization from answer generation, and provides a formal analysis grounded in associative memory theory. Evaluated across eight 128K-context datasets, the method achieves state-of-the-art average performance on Qwen3-4B, Qwen3-8B, and Llama3-8B, significantly improving evidence utilization.

Abstract

Understanding and reasoning over long contexts has become a key requirement for deploying large language models (LLMs) in realistic applications. Although recent LLMs support increasingly long context windows, they often fail to use relevant evidence that is already present in the input, revealing a gap between context access and effective context utilization. In this work, we propose Recursive Evidence Replay as LLM Harness for Long-Context Reasoning (RECONTEXT), a training-free inference method for improving long-context reasoning. RECONTEXT uses model-internal relevance signals to construct a query-conditioned evidence pool and replays it before final generation while preserving the full original context. This recursive selection process separates evidence organization from answer generation without training, external memory, or context pruning. We also provide a theoretical analysis based on associative memory, which characterizes the context as a memory store, the question as a retrieval cue, attention as cue-trace association, and replay as trace reactivation. Experiments on eight long-context datasets with 128K context length show that RECONTEXT consistently improves evidence utilization across Qwen3-4B, Qwen3-8B, and Llama3-8B, achieving the best average rank on all three backbones. Code is available at https://github.com/Yanjun-Zhao/ReContext.

Problem

• long-context reasoning
• evidence utilization
• large language models
• context access
• relevance

Innovation

• Recursive Evidence Replay
• Long-Context Reasoning
• Training-Free Inference
• Evidence Utilization
• Associative Memory