GRATE: Temporal Extensions for Inductive KG Foundation Models via Gated Rotary Attention

2026-07-11

AI Summary

This work addresses the challenge of inductive temporal knowledge graph reasoning across datasets with no overlap in entities, relations, or timestamps. We propose GRATE, a lightweight, parameter-free temporal encoding module that endows models with temporal modeling capabilities while preserving structural inductiveness. GRATE leverages Gated Rotary Attention to rotate edge representations according to relative time differences during message passing and employs query gating to filter temporally relevant signals. Integrated into the NBFNet architecture, GRATE effectively fuses temporal and structural information without introducing additional learnable parameters. We introduce GDELTIndT and WIKIIndT—the first benchmarks enabling cross-dataset inductive temporal transfer, supporting both interpolation and extrapolation settings. Experiments demonstrate that a single pretrained GRATE model significantly outperforms static baselines on most tasks.

Abstract

Knowledge graph foundation models such as Ultra and Trix achieve strong inductive transfer by learning relation-graph representations that generalise to unseen entities and relations. Extending this transferability to temporal knowledge graphs (TKGs) remains challenging: existing temporal models tie their parameters to dataset-specific entities, relations, or timestamps and are not designed to transfer to TKGs with disjoint vocabularies. We propose GRATE (Gated Rotary Attention for Temporal Encoding), an entity-side message function that adds no learnable parameters and encodes time through relative time differences by rotating each edge message according to its time gap to the query and applying a query-conditioned gate to select temporally relevant signals. GRATE integrates into NBFNet-style KG foundation models while preserving structural transferability. Existing TKG benchmarks evaluate within shared train/test vocabularies and cannot directly test cross-dataset temporal transfer; we therefore construct GDELTIndT and WIKIIndT, inductive transfer benchmark suites with disjoint entities, relations, and timestamps spanning both interpolation and extrapolation. Across these benchmarks and held-out forecasting datasets, a single jointly pretrained GRATE checkpoint improves over the static base model in most settings.

Problem

• temporal knowledge graphs
• inductive transfer
• knowledge graph foundation models
• cross-dataset transfer
• disjoint vocabularies

Innovation

• Gated Rotary Attention
• Temporal Knowledge Graphs
• Inductive Transfer
• Relative Time Encoding
• Foundation Models