🤖 AI Summary
This work addresses the challenges of catastrophic forgetting and high retraining costs in continual learning with large language models (LLMs). The authors propose modeling episodic memory as a dynamic temporal knowledge graph—where events serve as nodes and timestamped directed edges represent relations—as an external, updatable memory for a frozen LLM. During inference, retrieval-augmented generation (RAG) is enhanced by combining random walks with centrality-based retrieval from this graph. This approach effectively incorporates new knowledge without requiring model fine-tuning and, for the first time, demonstrates that structural properties of the graph—such as the clustering coefficient—can predict retrieval performance. Experiments show that, across three temporal recall tasks, the method reduces forgetting by approximately 7% compared to fine-tuning baselines and improves temporal ordering accuracy by about 5% over standard RAG.
📝 Abstract
Large Language Models (LLMs) struggle to incorporate new knowledge without forgetting or costly retraining. We propose DYNA, a lightweight framework that augments a frozen LLM with a temporal knowledge graph where events are nodes and temporal relations are directed, timestamped edges. The graph serves as an external, updatable memory. At query time, DYNA retrieves relevant nodes via random walks and centrality measures, then augments the LLM's response. Evaluated on three temporal recall tasks, DYNA reduces catastrophic forgetting by ~7% compared to fine-tuning and improves temporal ordering by ~5% over standard RAG. Higher graph clustering coefficients correlate with better retrieval, showing that graph structure matters. Contributions: (1) episodic memory as temporal KG, (2) retraining-free LLM augmentation, (3) graph properties as predictors of retrieval performance.