Temporal Knowledge Graph Forecasting under Distribution Shifts: A Synthetic Evaluation

2026-07-10

AI Summary

This study addresses the lack of systematic evaluation of robustness in temporal knowledge graph forecasting models under distribution shifts. The authors propose the first experimental framework based on controllable synthetic data, generating time structures characterized by repetitiveness, homogeneity, and periodicity, and simulating both static and diverse distribution shift scenarios to systematically assess seven representative model families. Their analysis reveals that simple memory-based models excel in data dominated by repetitiveness, while abrupt changes in entity community structure constitute the most challenging shift type. Overall model robustness is found to be highly dependent on the specific type of temporal regularity present in the data. This work exposes the strong reliance of current methods on particular temporal signals and establishes a new benchmark and set of insights for developing robust temporal reasoning approaches.

Abstract

Temporal knowledge graphs (TKGs) represent evolving relational systems, whose underlying data-generating processes often change over time. Yet, TKG forecasting models are commonly evaluated only on empirical benchmark datasets that provide limited insight into the models' robustness to such distribution shifts. Recognising this issue, we study TKG forecasting under controlled shift environments using a synthetic TKG generator that encodes three temporal and structural properties -- recurrence, homophily, and periodicity -- as data-generating mechanisms. This allows us to evaluate seven forecasting architectures under stationary and shifting regimes. Our experiments suggest that robustness in TKG forecasting is highly signal-dependent. Recurrence-based and periodic regularities are largely recoverable under stationary conditions, and simple memory-based baselines can be competitive when recurrence dominates the data. However, structural breaks reveal limitations in model adaptivity, with shifts in latent entity-community structure posing the strongest challenge in our study. Overall, our findings improve the understanding of the capabilities and limitations of current TKG models confronted with temporal distribution shifts.

Problem

• Temporal Knowledge Graph
• Distribution Shifts
• Forecasting
• Robustness
• Synthetic Evaluation

Innovation

• Temporal Knowledge Graph
• Distribution Shift
• Synthetic Data Generation
• Model Robustness
• Recurrence and Periodicity