Towards Improving Long-Tail Entity Predictions in Temporal Knowledge Graphs through Global Similarity and Weighted Sampling

2025-07-25

AI Summary

To address the poor generalization of temporal knowledge graph (TKG) completion models to unseen or sparsely connected novel entities, this paper proposes a generic incremental training framework. The method introduces a model-agnostic global similarity enhancement layer to overcome the local neighborhood constraints inherent in graph neural networks (GNNs), and designs a weighted sampling strategy tailored for long-tail entities to mitigate catastrophic forgetting. The framework seamlessly integrates with existing TKG completion models without architectural modifications. Evaluated on two benchmark datasets, it achieves absolute improvements of 10% and 15% in overall link prediction MRR, respectively, while significantly enhancing predictive performance for both novel and long-tail entities. The core contribution lies in the principled integration of global similarity modeling, sparsity-aware sampling, and incremental learning—constituting the first systematic solution to the dual challenges of generalization for dynamically emerging and sparsely connected entities in TKGs.

Technology Category

• Data Mining & Knowledge Management: Linked Open Data, Knowledge Graphs & KB Completion
• Machine Learning: Graph-based Machine Learning
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Temporal Knowledge Graph (TKG) completion models traditionally assume access to the entire graph during training. This overlooks challenges stemming from the evolving nature of TKGs, such as: (i) the model's requirement to generalize and assimilate new knowledge, and (ii) the task of managing new or unseen entities that often have sparse connections. In this paper, we present an incremental training framework specifically designed for TKGs, aiming to address entities that are either not observed during training or have sparse connections. Our approach combines a model-agnostic enhancement layer with a weighted sampling strategy, that can be augmented to and improve any existing TKG completion method. The enhancement layer leverages a broader, global definition of entity similarity, which moves beyond mere local neighborhood proximity of GNN-based methods. The weighted sampling strategy employed in training accentuates edges linked to infrequently occurring entities. We evaluate our method on two benchmark datasets, and demonstrate that our framework outperforms existing methods in total link prediction, inductive link prediction, and in addressing long-tail entities. Notably, our method achieves a 10% improvement and a 15% boost in MRR for these datasets. The results underscore the potential of our approach in mitigating catastrophic forgetting and enhancing the robustness of TKG completion methods, especially in an incremental training context

Problem

• Improving predictions for long-tail entities in Temporal Knowledge Graphs
• Addressing sparse connections and unseen entities in TKGs
• Enhancing robustness and reducing catastrophic forgetting in incremental training

Innovation

• Incremental training framework for TKGs
• Global entity similarity enhancement layer
• Weighted sampling for rare entities