Hindsight Memory-PRM: Supervising Memory Management with Auditable Hindsight Credit

2026-08-30

AI Summary

该研究针对长时序LLM代理的记忆操作难以监督的问题，提出了一种基于事后可审计信用的Hindsight Memory-PRM方法来优化记忆管理。

Abstract

Memory operations of long-horizon LLM agents are hard to supervise: an operation's value is unobservable when it is taken. But they are special -- they leave machine-readable evidence in the trajectory: retrieval hits and answer-time citations. Hindsight Memory-PRM exploits this audit trail twice: offline to train an operation-conditioned memory-utility critic, and online, where retrievals, citations, and one controlled deletion-and-reanswer per probe settle an intervention-calibrated entry-level presence credit, propagated along version chains as an action-level proxy reward -- no per-operation human labels, no Monte-Carlo replay of continuations. On held-out LoCoMo a local 8B policy reaches 77.5% under a fixed shared reader, surpassing its API teacher (65.1%) and all reproduced external systems, at one eighth the context of Mem0's official operating point; on LongMemEval, 79.0%. Ablations attribute the gain to causal calibration rather than signal density, and the policy converges to a multi-version memory organization whose gains no tested open-loop baseline reproduces.

Problem

• Memory Operations
• Long-horizon LLM Agents
• Supervision
• Unobservable Value
• Machine-readable Evidence

Innovation

• Hindsight Memory-PRM
• memory management
• auditable hindsight credit
• operation-conditioned memory-utility critic
• intervention-calibrated entry-level presence credit