From Passive Retrieval to Active Memory Navigation: Learning to Use Memory as a Structured Action Space

2026-07-06

AI Summary

This work addresses the limitation of existing personalized dialogue systems, which predominantly rely on passive retrieval of user memories and struggle to actively leverage long-term memory for reasoning. To overcome this, we propose NapMem, a novel framework that, for the first time, models long-term user memory as a learnable, structured action space. NapMem organizes memory content into a multi-granularity memory pyramid and introduces a memory tool interface, enabling the agent to dynamically select and navigate across memory granularities. Trained end-to-end with reinforcement learning, the agent learns to proactively invoke relevant memories during response generation. Experimental results demonstrate that NapMem significantly outperforms baseline methods on memory-intensive benchmarks—including PersonaMem-v2, LongMemEval, and LoCoMo—while maintaining strong general reasoning and tool-use capabilities on non-memory tasks.

Abstract

Long-term user memory is essential for personalized conversational agents, yet many memory systems still expose memory through passive retrieval interfaces, making the model a consumer of pre-selected evidence. We introduce NapMem, a framework for learning to use long-term user memory as a structured action space rather than passively retrieved context. NapMem organizes user history into a linked multi-granularity memory pyramid, where raw conversations, typed memory records, topic tracks, and user profiles are connected through provenance relations, and exposes these levels through memory tools. The agent is trained to select memory according to the query and intermediate evidence, allowing it to inspect different memory granularities before answering. Experiments on PersonaMem-v2, LongMemEval, and LoCoMo show that a NapMem agent trained with memory-tool reinforcement learning is competitive across diverse memory-intensive tasks, while evaluations on non-memory tasks suggest that the learned policy largely preserves general reasoning and tool-use abilities. Additional analyses examine storage, inference cost, tool-use behavior, and ablations over navigation, memory granularity, and RL training. Our results suggest that long-term user memory benefits from coupling structured storage with a learned policy for using memory at the appropriate granularity.

Problem

• long-term memory
• personalized conversational agents
• memory retrieval
• memory granularity
• structured action space

Innovation

• structured memory
• memory navigation
• reinforcement learning
• multi-granularity memory
• tool-augmented reasoning