Agentic Memory: Learning Unified Long-Term and Short-Term Memory Management for Large Language Model Agents

2026-01-05

arXiv.org

AI Summary

This work addresses the limitations of large language model agents in long-horizon reasoning, which stem from constrained context windows and the disjointed treatment of short- and long-term memory in existing approaches, lacking unified optimization and adaptability. To overcome these challenges, we propose the Agentic Memory (AgeMem) framework, which, for the first time, formulates memory management as learnable tool-augmented actions within the agent’s policy. This enables autonomous storage, retrieval, updating, and discarding of memories through a tool-based mechanism. We further introduce a three-stage progressive reinforcement learning curriculum combined with a stepwise GRPO algorithm to effectively mitigate sparse reward issues. Experimental results demonstrate that AgeMem significantly outperforms strong baselines across five long-horizon task benchmarks, achieving higher task completion rates, improved memory quality, and more efficient context utilization.

Technology Category

• Multiagent Systems: Multiagent Learning
• Cognitive Modeling & Cognitive Systems: Agent Architectures
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Agentic search
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Large language model (LLM) agents face fundamental limitations in long-horizon reasoning due to finite context windows, making effective memory management critical. Existing methods typically handle long-term memory (LTM) and short-term memory (STM) as separate components, relying on heuristics or auxiliary controllers, which limits adaptability and end-to-end optimization. In this paper, we propose Agentic Memory (AgeMem), a unified framework that integrates LTM and STM management directly into the agent's policy. AgeMem exposes memory operations as tool-based actions, enabling the LLM agent to autonomously decide what and when to store, retrieve, update, summarize, or discard information. To train such unified behaviors, we propose a three-stage progressive reinforcement learning strategy and design a step-wise GRPO to address sparse and discontinuous rewards induced by memory operations. Experiments on five long-horizon benchmarks demonstrate that AgeMem consistently outperforms strong memory-augmented baselines across multiple LLM backbones, achieving improved task performance, higher-quality long-term memory, and more efficient context usage.

Problem

• long-horizon reasoning
• memory management
• large language model agents
• long-term memory
• short-term memory

Innovation

• Agentic Memory
• unified memory management
• tool-based memory actions
• progressive reinforcement learning
• long-horizon reasoning