Learning to Continually Learn via Meta-learning Agentic Memory Designs

2026-02-08

AI Summary

This work addresses the limitation of stateless foundation models in enabling continual learning for agents during testing, as well as the difficulty of handcrafted memory modules to adapt to task diversity and non-stationarity. To overcome these challenges, the authors propose ALMA, a framework that, for the first time, models memory structures as a learnable program space represented in executable code. ALMA employs meta-learning to automatically search over memory designs—including database schemas, retrieval strategies, and update mechanisms—eliminating the need for manual customization and enabling adaptive continual learning across tasks. Experiments on four sequential decision-making tasks demonstrate that the memory mechanisms discovered by ALMA consistently outperform state-of-the-art hand-designed approaches in both learning efficiency and task performance.

Technology Category

• Machine Learning: Life-Long and Continual Learning
• Multiagent Systems: Multiagent Learning
• Search and Optimization: Learning to Search

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

The statelessness of foundation models bottlenecks agentic systems'ability to continually learn, a core capability for long-horizon reasoning and adaptation. To address this limitation, agentic systems commonly incorporate memory modules to retain and reuse past experience, aiming for continual learning during test time. However, most existing memory designs are human-crafted and fixed, which limits their ability to adapt to the diversity and non-stationarity of real-world tasks. In this paper, we introduce ALMA (Automated meta-Learning of Memory designs for Agentic systems), a framework that meta-learns memory designs to replace hand-engineered memory designs, therefore minimizing human effort and enabling agentic systems to be continual learners across diverse domains. Our approach employs a Meta Agent that searches over memory designs expressed as executable code in an open-ended manner, theoretically allowing the discovery of arbitrary memory designs, including database schemas as well as their retrieval and update mechanisms. Extensive experiments across four sequential decision-making domains demonstrate that the learned memory designs enable more effective and efficient learning from experience than state-of-the-art human-crafted memory designs on all benchmarks. When developed and deployed safely, ALMA represents a step toward self-improving AI systems that learn to be adaptive, continual learners.

Problem

• continual learning
• agentic systems
• memory design
• meta-learning
• non-stationarity

Innovation

• meta-learning
• agentic memory
• continual learning
• automated design
• executable code