WhenLoss: Diagnosing Write and Retrieval Bottlenecks in Long-Context Memory Systems

2026-05-23

AI Summary

Existing long-context memory systems often suffer performance degradation under fixed budgets due to either write-time compression or retrieval failure, yet lack effective means to disentangle these two bottlenecks. This work introduces a four-condition diagnostic protocol—comprising Task-Focused Context (TFC), Oracle Extraction (OE), Compression Sufficiency Metric (CSM), and Retrieval Match (RM)—to quantitatively isolate and assess write versus retrieval limitations, revealing that most systems are primarily constrained during the write phase. Building on this insight, we propose a novel paradigm termed Expected Predictive Compression (EPC), which leverages large language models at write time to anticipate future queries and retain only the minimal necessary evidence, leaving the retrieval mechanism unchanged. Evaluated on 500 questions from LongMemEval, EPC achieves a CSM score of 0.49, outperforming the strongest baseline (0.44) and reducing the write gap Δ_write to 0.04.

Abstract

Long-context memory systems often fail under fixed budgets, but end-to-end evaluation does not reveal whether evidence was discarded during compression or preserved but never retrieved. We introduce a four-condition diagnostic protocol that evaluates a fixed reader under truncated full context (TFC), oracle evidence (OE), complete stored memory (CSM), and retrieved memory (RM). Under this fixed-budget LongMemEval setup, write-side gaps exceed retrieval-side gaps for most tested baselines, with four of six baselines robustly write-dominant under our default diagnosis margin. Motivated by this diagnosis, we propose Expected Predictive Compression (EPC), which moves the key decision--what information to retain--to write time by using an LLM to anticipate likely future questions and preserve the minimal supporting evidence under the token budget, while leaving retrieval unchanged at question time. Across all 500 LongMemEval questions with three readers (GPT-5.2, Claude Sonnet 4, Gemini 2.5 Pro), EPC achieves the highest CSM scores among all systems (0.49 vs. 0.44 for Summary (LLM), the strongest baseline), reducing Delta_write to 0.04 while leaving Delta_retr comparable to other LLM-based systems. These results suggest that, on this benchmark and evaluation setup, improving what the write stage preserves is a key avenue for performance gains in the tested systems.

Problem

• long-context memory
• write bottleneck
• retrieval bottleneck
• memory compression
• evidence preservation

Innovation

• Expected Predictive Compression
• long-context memory
• write-retrieval bottleneck diagnosis
• fixed-budget compression
• evidence preservation