Remember Me, Refine Me: A Dynamic Procedural Memory Framework for Experience-Driven Agent Evolution

2025-12-11

AI Summary

Existing programmatic memory frameworks predominantly adopt static, append-only archival paradigms, limiting their capacity for dynamic reasoning and experience evolution. This paper introduces ReMe, a dynamic programmatic memory framework that pioneers end-to-end memory lifecycle management—encompassing multi-dimensional experience distillation, scenario-aware index reuse, and utility-driven automatic memory insertion, deletion, and revision—to bridge static storage and dynamic inference in a closed loop. Key technical innovations include failure attribution analysis, contrastive insight generation, context-adaptive retrieval, and online memory refinement. ReMe achieves state-of-the-art performance on BFCL-V3 and AppWorld. Notably, Qwen3-8B augmented with ReMe significantly outperforms the memory-free Qwen3-14B, empirically validating a novel paradigm: lightweight models can attain effective lifelong learning through efficient, adaptive memory mechanisms.

Technology Category

• Machine Learning: Life-Long and Continual Learning
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Procedural memory enables large language model (LLM) agents to internalize "how-to" knowledge, theoretically reducing redundant trial-and-error. However, existing frameworks predominantly suffer from a "passive accumulation" paradigm, treating memory as a static append-only archive. To bridge the gap between static storage and dynamic reasoning, we propose $ extbf{ReMe}$ ($ extit{Remember Me, Refine Me}$), a comprehensive framework for experience-driven agent evolution. ReMe innovates across the memory lifecycle via three mechanisms: 1) $ extit{multi-faceted distillation}$, which extracts fine-grained experiences by recognizing success patterns, analyzing failure triggers and generating comparative insights; 2) $ extit{context-adaptive reuse}$, which tailors historical insights to new contexts via scenario-aware indexing; and 3) $ extit{utility-based refinement}$, which autonomously adds valid memories and prunes outdated ones to maintain a compact, high-quality experience pool. Extensive experiments on BFCL-V3 and AppWorld demonstrate that ReMe establishes a new state-of-the-art in agent memory system. Crucially, we observe a significant memory-scaling effect: Qwen3-8B equipped with ReMe outperforms larger, memoryless Qwen3-14B, suggesting that self-evolving memory provides a computation-efficient pathway for lifelong learning. We release our code and the $ exttt{reme.library}$ dataset to facilitate further research.

Problem

• Addresses static procedural memory in LLM agents
• Introduces dynamic memory lifecycle for agent evolution
• Enables efficient lifelong learning through self-evolving memory

Innovation

• Multi-faceted distillation extracts fine-grained experiences from patterns and failures
• Context-adaptive reuse tailors historical insights to new scenarios via indexing
• Utility-based refinement autonomously adds valid memories and prunes outdated ones