TGFormer: Towards Temporal Graph Transformer with Auto-Correlation Mechanism

2026-05-24

AI Summary

Existing approaches struggle to effectively model long-range dependencies and periodic interactions in temporal graphs. This work proposes a novel Temporal Graph Transformer architecture that constructs node representations through trajectory modeling and historical interaction analysis, and innovatively introduces an autocorrelation mechanism grounded in stochastic process theory to explicitly capture periodicity and long-range dependencies at the sub-interaction level. By transcending the limitations of conventional attention mechanisms, the proposed method achieves significant performance gains over state-of-the-art models across six public benchmarks, with improvements in prediction accuracy reaching up to 9.35%.

Abstract

The growing interest in Temporal Graph Neural Networks (TGNNs) stems from their ability to model complex dynamics and deliver superior performance. However, TGNNs encounter fundamental challenges in capturing long-term dependencies and identifying periodic patterns. To address these limitations, we propose TGFormer, a novel Transformer architecture specifically designed for temporal graphs. Our model redefines temporal graph learning by establishing a trajectory framework that aligns with time series analysis principles. This approach allows TGFormer to derive node representations through systematic analysis of historical interactions, enabling granular examination of node relationships across sequential timestamps. Building upon stochastic process theory, we develop an auto-correlation mechanism that systematically uncovers periodic dependencies in node interactions. This innovation empowers TGFormer to perform dependency discovery and representation aggregation at sub-interaction levels, demonstrating superior efficiency and accuracy compared to conventional attention mechanisms. Experimental validation across six public benchmarks confirms the effectiveness of our approach, with TGFormer at most achieving 9.35\% precision improvement compared to state-of-the-art approaches.

Problem

• Temporal Graph Neural Networks
• long-term dependencies
• periodic patterns
• temporal graphs
• node interactions

Innovation

• Temporal Graph Transformer
• Auto-Correlation Mechanism
• Trajectory Framework
• Periodic Dependencies
• Sub-Interaction Aggregation