Revisiting Catastrophic Forgetting in Continual Knowledge Graph Embedding

2026-04-21

AI Summary

This work addresses a critical oversight in existing continual knowledge graph embedding methods: the neglect of interference from newly introduced entity embeddings on predictions involving previously seen entities, which leads to substantial overestimation of model performance. The study is the first to identify and formally define this “entity interference” phenomenon and introduces a revised evaluation protocol along with dedicated metrics for measuring forgetting in continual knowledge graph learning. Through comprehensive experiments across multiple benchmarks, the authors systematically analyze how state-of-the-art approaches are affected by such interference. Their findings reveal that ignoring entity interference can inflate performance estimates by up to 25%, particularly undermining evaluation reliability in scenarios with rapid entity growth. This work thus establishes a more accurate foundation for evaluating continual knowledge graph embedding methods.

Technology Category

• Data Mining & Knowledge Management: Linked Open Data, Knowledge Graphs & KB Completion
• Knowledge Representation and Reasoning: Other Foundations of Knowledge Representation & Reasoning
• Machine Learning: Life-Long and Continual Learning

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

Knowledge Graph Embeddings (KGEs) support a wide range of downstream tasks over Knowledge Graphs (KGs). In practice, KGs evolve as new entities and facts are added, motivating Continual Knowledge Graph Embedding (CKGE) methods that update embeddings over time. Current CKGE approaches address catastrophic forgetting (i.e., the performance degradation on previously learned tasks) primarily by limiting changes to existing embeddings. However, we show that this view is incomplete. When new entities are introduced, their embeddings can interfere with previously learned ones, causing the model to predict them in place of previously correct answers. This phenomenon, which we call entity interference, has been largely overlooked and is not accounted for in current CKGE evaluation protocols. As a result, the assessment of catastrophic forgetting becomes misleading, and CKGE methods performance is systematically overestimated. To address this issue, we introduce a corrected CKGE evaluation protocol that accounts for entity interference. Through experiments on multiple benchmarks, we show that ignoring this effect can lead to performance overestimation of up to 25%, particularly in scenarios with significant entity growth. We further analyze how different CKGE methods and KGE models are affected by the different sources of forgetting, and introduce a catastrophic forgetting metric tailored to CKGE.

Problem

• catastrophic forgetting
• continual knowledge graph embedding
• entity interference
• knowledge graph embeddings
• evaluation protocol

Innovation

• catastrophic forgetting
• continual knowledge graph embedding
• entity interference
• evaluation protocol
• knowledge graph embeddings