Diagnosing and Mitigating Context Rot in Long-horizon Search

2026-06-28

AI Summary

This study addresses the phenomenon of “context corruption” in large language models during long-horizon search tasks, wherein extended input contexts lead models to prematurely abandon reasoning or produce uncertain outputs. The authors systematically evaluate multiple open-source large language models in deep search scenarios and, for the first time, establish a clear link between increasing context length and model abandonment behavior. To mitigate this issue, they propose an integrated approach combining context pruning, seven distinct context management strategies, and corruption-aware posterior rejection sampling, further enhanced by a multi-model aggregation mechanism. Experimental results demonstrate that the proposed method substantially alleviates performance degradation, with the combined strategy achieving state-of-the-art effectiveness in long-horizon search tasks.

Abstract

Extensive context has become the norm as Large Language Models (LLMs) are increasingly deployed in long-horizon tasks. The concern that increasing context length degrades model capabilities, known as context rot, has become a central issue for these applications. In this paper, we focus on deep search scenarios, aiming to investigate the rot phenomenon and its mitigation strategies. By evaluating four flagship open-source models across three benchmarks, we reveal a prevalent but unnoticed rot phenomenon: extensive context causes models to directly give up or prematurely provide uncertain answers, and this issue is exacerbated as the context grows. Through pruning experiments, we demonstrate the relationship between the accumulated context and the rot phenomenon. Furthermore, we investigate mitigating this issue through context management and post-hoc rejection sampling. For context management, we systematically evaluate seven different methods across three categories, based on performance, cost, and impact on context rot, providing clear guidance for strategy selection and usage. For rejection sampling, we develop a rot-aware filtering strategy and demonstrate its effectiveness across three aggregation methods. Finally, we show that these two approaches can be combined for further performance improvements.

Problem

• context rot
• long-horizon tasks
• large language models
• extensive context
• model degradation

Innovation

• context rot
• long-horizon search
• context management
• rejection sampling
• large language models