StreamTGN: A GPU-Efficient Serving System for Streaming Temporal Graph Neural Networks

2026-03-22

AI Summary

This work addresses the severe computational redundancy in existing temporal graph neural network (GNN) inference systems, which trigger full-graph embedding updates upon the arrival of new edges. The paper proposes the first efficient inference system tailored for streaming temporal GNNs, leveraging the locality of graph updates to refresh only affected nodes. Key innovations include a dirty-label propagation mechanism for identifying impacted nodes, a drift-aware adaptive reconstruction scheduler, and a batching strategy that relaxes strict event-ordering constraints. Built upon a GPU-resident node memory and streaming processing architecture, the system achieves 4.5–739× speedup for TGN inference and up to 4207× for TGAT across eight temporal graphs, with end-to-end training-inference co-optimization yielding up to 24× acceleration—all without sacrificing model accuracy.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Data management and stream processing for Web, mobile and wireless applications
• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies

Abstract

Temporal Graph Neural Networks (TGNs) achieve state-of-the-art performance on dynamic graph tasks, yet existing systems focus exclusively on accelerating training -- at inference time, every new edge triggers $O(|V|)$ embedding updates even though only a small fraction of nodes are affected. We present \textbf{StreamTGN}, the first streaming TGN inference system exploiting the inherent locality of temporal graph updates: in an $L$-layer TGN, a new edge affects only nodes within $L$ hops of the endpoints, typically less than 0.2\% on million-node graphs. StreamTGN maintains persistent GPU-resident node memory and uses dirty-flag propagation to identify the affected set $\mathcal{A}$, reducing per-batch complexity from $O(|V|)$ to $O(|\mathcal{A}|)$ with zero accuracy loss. Drift-aware adaptive rebuild scheduling and batched streaming with relaxed ordering further maximize throughput. Experiments on eight temporal graphs (2K--2.6M nodes) show 4.5$\times$--739$\times$ speedup for TGN and up to 4,207$\times$ for TGAT, with identical accuracy. StreamTGN is orthogonal to training optimizations: combining SWIFT with StreamTGN yields 24$\times$ end-to-end speedup across three architectures (TGN, TGAT, DySAT).

Problem

• Temporal Graph Neural Networks
• streaming inference
• dynamic graphs
• GPU efficiency
• embedding updates

Innovation

• Temporal Graph Neural Networks
• Streaming Inference
• Locality-aware Update
• GPU-efficient System
• Dirty-flag Propagation