The AI Hippocampus: How Far are We From Human Memory?

2026-01-14

Trans. Mach. Learn. Res.

AI Summary

This work addresses the construction of human-like memory mechanisms in large language models and multimodal foundation models to support continual learning, personalized reasoning, and cross-modal consistency. It proposes the first unified taxonomic framework that systematically integrates three major memory paradigms: implicit memory (e.g., parameterized memory), explicit memory (e.g., external retrieval and graph-structured knowledge bases), and agent memory. The framework is further extended to multimodal settings, elucidating the critical role of memory in cross-modal alignment and agent collaboration. Through a comprehensive literature review and taxonomic analysis, the study surveys existing technical approaches, evaluation benchmarks, and core challenges, thereby establishing a theoretical foundation and offering clear directions for future research on human-like memory systems in artificial intelligence.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Cognitive Modeling & Cognitive Systems: Agent Architectures
• Multiagent Systems: Multiagent Learning

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Memory plays a foundational role in augmenting the reasoning, adaptability, and contextual fidelity of modern Large Language Models and Multi-Modal LLMs. As these models transition from static predictors to interactive systems capable of continual learning and personalized inference, the incorporation of memory mechanisms has emerged as a central theme in their architectural and functional evolution. This survey presents a comprehensive and structured synthesis of memory in LLMs and MLLMs, organizing the literature into a cohesive taxonomy comprising implicit, explicit, and agentic memory paradigms. Specifically, the survey delineates three primary memory frameworks. Implicit memory refers to the knowledge embedded within the internal parameters of pre-trained transformers, encompassing their capacity for memorization, associative retrieval, and contextual reasoning. Recent work has explored methods to interpret, manipulate, and reconfigure this latent memory. Explicit memory involves external storage and retrieval components designed to augment model outputs with dynamic, queryable knowledge representations, such as textual corpora, dense vectors, and graph-based structures, thereby enabling scalable and updatable interaction with information sources. Agentic memory introduces persistent, temporally extended memory structures within autonomous agents, facilitating long-term planning, self-consistency, and collaborative behavior in multi-agent systems, with relevance to embodied and interactive AI. Extending beyond text, the survey examines the integration of memory within multi-modal settings, where coherence across vision, language, audio, and action modalities is essential. Key architectural advances, benchmark tasks, and open challenges are discussed, including issues related to memory capacity, alignment, factual consistency, and cross-system interoperability.

Problem

• memory
• Large Language Models
• Multi-Modal LLMs
• continual learning
• autonomous agents

Innovation

• memory taxonomy
• implicit memory
• explicit memory
• agentic memory
• multi-modal memory