All-Mem: Agentic Lifelong Memory via Dynamic Topology Evolution

2026-03-19

AI Summary

This work addresses the degradation in retrieval quality faced by lifelong interactive agents due to ever-growing memory stores plagued by redundancy, obsolescence, and noise. To tackle this challenge, the authors propose All-Mem, a framework that maintains a structured memory repository through dynamic topological evolution. During online interaction, retrieval is anchored to a bounded visible surface, while offline, a large language model–based diagnostic module drives non-destructive topological edits—such as SPLIT, MERGE, and UPDATE—augmented with confidence scores, all while preserving immutable evidence for traceability. Innovatively integrating a hybrid architecture, typed links, and hop-constrained retrieval, All-Mem circumvents the information loss inherent in conventional summarization-based compression. Evaluated on the LOCOMO and LONGMEMEVAL benchmarks, All-Mem significantly outperforms existing approaches, achieving consistent gains in both retrieval accuracy and downstream question-answering performance.

Technology Category

• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Machine Learning: Large Multimodal Models (LMMs)
• Cognitive Modeling & Cognitive Systems: Agent Architectures

Application Category

• Search and Retrieval-Augmented AI: Agentic search
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Lifelong interactive agents are expected to assist users over months or years, which requires continually writing long term memories while retrieving the right evidence for each new query under fixed context and latency budgets. Existing memory systems often degrade as histories grow, yielding redundant, outdated, or noisy retrieved contexts. We present All-Mem, an online/offline lifelong memory framework that maintains a topology structured memory bank via explicit, non destructive consolidation, avoiding the irreversible information loss typical of summarization based compression. In online operation, it anchors retrieval on a bounded visible surface to keep coarse search cost bounded. Periodically offline, an LLM diagnoser proposes confidence scored topology edits executed with gating using three operators: SPLIT, MERGE, and UPDATE, while preserving immutable evidence for traceability. At query time, typed links enable hop bounded, budgeted expansion from active anchors to archived evidence when needed. Experiments on LOCOMO and LONGMEMEVAL show improved retrieval and QA over representative baselines.

Problem

• lifelong memory
• memory retrieval
• interactive agents
• context budget
• latency constraint

Innovation

• lifelong memory
• dynamic topology evolution
• non-destructive consolidation
• typed links
• LLM-guided memory editing