DepTGL: A Parallel Framework for Memory-based TGNN Training with Adaptive Temporal Data Dependency Management

2026-08-17

AI Summary

This study addresses the challenges of missing temporal dependencies, high synchronization overhead, and load imbalance in the distributed training of memory-based Temporal Graph Neural Networks (M-TGNNs). To overcome these limitations, we propose DepTGL, a novel framework featuring a hybrid temporal dependency management mechanism. By integrating gradient-aware cache synchronization with load-aware temporal pruning, DepTGL reconstructs dependency management from a data-centric perspective to accelerate training. Extensive experiments on six real-world datasets demonstrate that DepTGL achieves an average training speedup of 4.99× while preserving model accuracy. These results confirm that the proposed framework effectively mitigates critical performance bottlenecks in M-TGNN distributed training, offering a robust solution for scaling temporal graph learning without compromising predictive performance.

Abstract

Memory-based Temporal Graph Neural Networks (M-TGNNs) maintain recursively updated node states to capture fine-grained temporal interactions. However, existing distributed frameworks lack effective mechanisms for managing the temporal data dependencies inherent in these models. As a result, they must enforce strict chronological updates, incur substantial remote synchronization overhead, and experience severe load imbalance when temporal event streams are skewed. We propose DepTGL, a scalable distributed training framework that restructures temporal-dependency management for M-TGNNs from a data-centric perspective. First, DepTGL introduces a hybrid temporal-dependency management scheme that explicitly balances communication and caching overhead via temporal-event caching, supplemented by selective dependency-driven communication. Next, DepTGL incorporates a gradient-aware cache-synchronization policy that adaptively suppresses boundary updates as model optimization stabilizes, thereby reducing redundant synchronization. Finally, DepTGL integrates a load-aware temporal-pruning strategy that eliminates auxiliary replay events under skew-induced load spikes, reducing redundant data processing and mitigating straggler effects. Experiments on six real-world temporal graphs show that DepTGL achieves an average speedup of 4.99x over state-of-the-art baselines, while maintaining comparable accuracy.

Problem

• Memory-based TGNN
• Distributed Training
• Temporal Data Dependency
• Synchronization Overhead
• Load Imbalance

Innovation

• Memory-based TGNN
• Temporal Dependency Management
• Gradient-aware Cache Synchronization
• Load-aware Temporal Pruning
• Distributed Training