Personalize-then-Store: Benchmarking and Learning Personalized Memory for Long-horizon Agents

2026-05-25

AI Summary

Current large language models typically employ static, generic memory strategies that overlook inter-user variations in memory value, leading to inefficient memory utilization and poor support for long-horizon tasks. To address this limitation, this work introduces PerMemBench, the first benchmark tailored for evaluating personalized memory systems, and proposes a lightweight, conversation-level memory gating mechanism that enables models to learn to skip memorization of low-value interactions. Experimental results demonstrate that, under ideal gating conditions, personalized memory strategies substantially enhance the model’s ability to retain information over extended contexts. However, the findings also underscore that achieving precise gating remains a critical challenge for realizing the full potential of such approaches.

Abstract

Existing large language model (LLM) based memory systems apply universal, static policies that overlook a fundamental reality: the contexts that are worth storing in memory are different across users. This misalignment wastes limited memory budget on transient interactions while failing to preserve critical context for long horizon tasks. To address this gap, we investigate an underexplored question: can LLM based memory systems learn personalized memory policies? We introduce PerMemBench, the first benchmark for evaluating personalized memory systems, featuring multi year, multi domain interaction histories across diverse user personas. We further present the first empirical study of memory personalization, proposing session level storage gating, a lightweight framework that selectively bypasses memory operations for transient sessions. Our study confirms that personalization yields substantial retention gains under perfect gating, yet reveals that accurate gating remains an open and critical challenge.

Problem

• personalized memory
• long-horizon agents
• memory policy
• LLM-based memory systems
• memory budget

Innovation

• personalized memory
• memory gating
• long-horizon agents
• PerMemBench
• session-level storage