Towards Foundation Model on Temporal Knowledge Graph Reasoning

2025-06-04

AI Summary

Existing temporal knowledge graph embedding (TKGE) models rely on entities, relations, and timestamps observed during training, hindering generalization to unseen domains and limiting real-world deployment. Method: This paper proposes the first fully inductive TKGE framework for zero-shot link prediction. It introduces a time-agnostic adaptive representation generation mechanism that decouples semantic and temporal modeling; incorporates sinusoidal positional encoding and local-global temporal-aware message passing to enable cross-granularity and cross-span temporal structure transfer; and adopts a pretraining-finetuning paradigm to build a foundational TKGE model. Contribution/Results: Theoretical analysis and extensive experiments demonstrate that a single pretrained model achieves strong zero-shot performance across multiple unseen temporal knowledge graphs, significantly improving generalization to novel entities, relations, and timestamps—without requiring task-specific retraining or fine-tuning.

Technology Category

• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Data Mining & Knowledge Management: Mining of Spatial, Temporal or Spatio-Temporal Data
• Planning, Routing, and Scheduling: Temporal Planning

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Temporal Knowledge Graphs (TKGs) store temporal facts with quadruple formats (s, p, o, t). Existing Temporal Knowledge Graph Embedding (TKGE) models perform link prediction tasks in transductive or semi-inductive settings, which means the entities, relations, and temporal information in the test graph are fully or partially observed during training. Such reliance on seen elements during inference limits the models' ability to transfer to new domains and generalize to real-world scenarios. A central limitation is the difficulty in learning representations for entities, relations, and timestamps that are transferable and not tied to dataset-specific vocabularies. To overcome these limitations, we introduce the first fully-inductive approach to temporal knowledge graph link prediction. Our model employs sinusoidal positional encodings to capture fine-grained temporal patterns and generates adaptive entity and relation representations using message passing conditioned on both local and global temporal contexts. Our model design is agnostic to temporal granularity and time span, effectively addressing temporal discrepancies across TKGs and facilitating time-aware structural information transfer. As a pretrained, scalable, and transferable model, POSTRA demonstrates strong zero-shot performance on unseen temporal knowledge graphs, effectively generalizing to novel entities, relations, and timestamps. Extensive theoretical analysis and empirical results show that a single pretrained model can improve zero-shot performance on various inductive temporal reasoning scenarios, marking a significant step toward a foundation model for temporal KGs.

Problem

• Overcoming reliance on seen elements in TKGE models
• Learning transferable representations for entities and relations
• Generalizing to novel entities and timestamps in TKGs

Innovation

• Uses sinusoidal encodings for temporal patterns
• Adaptive entity-relation representations via message passing
• Pretrained scalable model for zero-shot transfer