🤖 AI Summary
This work addresses the challenge faced by large language model (LLM) memory systems in retrieving the most up-to-date and accurate facts when confronted with temporally evolving contradictory information. To resolve this, the authors propose a deterministic memory conflict resolution method that explicitly identifies the bottleneck in the post-retrieval assembly phase. Instead of relying on LLM-based judgment, the approach employs deterministic aggregation strategies—such as max(serial) or max(timestamp)—after candidate fact extraction. Integrated with Self-Ask multi-hop reasoning and a unified retrieval chunking mechanism, the method forms an end-to-end optimized pipeline. Experimental results demonstrate significant improvements: achieving 78.0% accuracy with gpt-4o-mini and 94.8% with gpt-4o on the single-hop FactConsolidation task, and 51.5% on the multi-hop setting with gpt-4o, outperforming existing systems by 20–28 percentage points.
📝 Abstract
LLM-based memory systems increasingly maintain facts that evolve over time, where a recurring failure is conflict resolution: when a fact has multiple contradictory values, which should the agent return? MemoryAgentBench (MAB; Hu et al., 2026) makes this explicit in its FactConsolidation task: facts are numbered, the counterfactual has the higher serial, and agents are told newer facts have larger serials. Yet every published system underperforms: HippoRAG-v2 reaches 54% on single-hop (FC-SH), BM25 48%, Mem0 18%, and the temporal KG Zep/Graphiti just 7%. Multi-hop is near-unsolved (at most 7% across 22 systems).
We argue the bottleneck is the assembly step: baselines leave conflict resolution to LLM-mediated retrieval or generation rather than version-aware aggregation. A matched-setup comparison (same backbone, retrieval, chunking, TOP_K) shows that replacing the LLM-judgment answer pipeline with candidate-extraction plus Python max(serial) yields +10.8 points on FC-SH (gpt-4o-mini), widening from +8 at 6K to +21 at 262K. This is a whole-pipeline effect (resolver, prompt, format, and temperature vary jointly); isolating the resolver is future work. The recipe reaches 78.0% on FC-SH (gpt-4o-mini), 94.8% (gpt-4o), and 30.2% on FC-MH (gpt-4o-mini, rising to 51.5% with gpt-4o) via a per-hop deterministic extension of Self-Ask. At matched-262K, it beats HippoRAG-v2 by +28 points and the best published FC-MH result by +20.
The implication is corrective for the subfield: the bottleneck on conflict resolution is assembly (post-retrieval aggregation), not storage. A LongMemEval knowledge-update check shows the mechanism ports from max(serial) to max(timestamp) but only ties LLM judgment (57.8% vs 64.4%, n=45): deterministic aggregation is the right primitive for current-value conflicts and must be composed with question-type-aware handling for broader memory QA.