Modular Memory is the Key to Continual Learning Agents

2026-03-02

AI Summary

Current foundation models exhibit significant limitations in continual learning, experience accumulation, and personalization, with conventional weight-update-based approaches particularly prone to catastrophic forgetting. This work proposes a novel architecture centered on modular memory that systematically integrates in-weight learning (IWL) and in-context learning (ICL) for the first time: ICL enables rapid acquisition of new knowledge, while IWL provides stable, long-term capability enhancement, and modular memory serves as a synergistic bridge between the two. The resulting framework offers an innovative and practical pathway toward building agents capable of lifelong evolution, continuous adaptation, and personalized interaction.

Technology Category

• Machine Learning: Life-Long and Continual Learning
• Cognitive Modeling & Cognitive Systems: Agent Architectures
• Multiagent Systems: Multiagent Learning

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Foundation models have transformed machine learning through large-scale pretraining and increased test-time compute. Despite surpassing human performance in several domains, these models remain fundamentally limited in continuous operation, experience accumulation, and personalization, capabilities that are central to adaptive intelligence. While continual learning research has long targeted these goals, its historical focus on in-weight learning (IWL), i.e., updating a single model's parameters to absorb new knowledge, has rendered catastrophic forgetting a persistent challenge. Our position is that combining the strengths of In-Weight Learning (IWL) and the newly emerged capabilities of In-Context Learning (ICL) through the design of modular memory is the missing piece for continual adaptation at scale. We outline a conceptual framework for modular memory-centric architectures that leverage ICL for rapid adaptation and knowledge accumulation, and IWL for stable updates to model capabilities, charting a practical roadmap toward continually learning agents.

Problem

• Continual Learning
• Catastrophic Forgetting
• In-Weight Learning
• In-Context Learning
• Modular Memory

Innovation

• Modular Memory
• Continual Learning
• In-Context Learning
• In-Weight Learning
• Adaptive Intelligence