Search papers, labs, and topics across Lattice.
This paper introduces DepTGL, a novel distributed training framework for Memory-based Temporal Graph Neural Networks (M-TGNNs) that addresses the challenges of managing temporal data dependencies. By implementing a hybrid temporal-dependency management scheme and a gradient-aware cache-synchronization policy, DepTGL significantly reduces synchronization overhead and enhances load balancing during training. Experimental results demonstrate that DepTGL achieves an impressive average speedup of 4.99x compared to existing state-of-the-art methods while preserving accuracy levels.
Achieving nearly 5x faster training for M-TGNNs without sacrificing accuracy could revolutionize how we handle temporal data in graph neural networks.
Memory-based Temporal Graph Neural Networks (M-TGNNs) maintain recursively updated node states to capture fine-grained temporal interactions. However, existing distributed frameworks lack effective mechanisms for managing the temporal data dependencies inherent in these models. As a result, they must enforce strict chronological updates, incur substantial remote synchronization overhead, and experience severe load imbalance when temporal event streams are skewed. We propose DepTGL, a scalable distributed training framework that restructures temporal-dependency management for M-TGNNs from a data-centric perspective. First, DepTGL introduces a hybrid temporal-dependency management scheme that explicitly balances communication and caching overhead via temporal-event caching, supplemented by selective dependency-driven communication. Next, DepTGL incorporates a gradient-aware cache-synchronization policy that adaptively suppresses boundary updates as model optimization stabilizes, thereby reducing redundant synchronization. Finally, DepTGL integrates a load-aware temporal-pruning strategy that eliminates auxiliary replay events under skew-induced load spikes, reducing redundant data processing and mitigating straggler effects. Experiments on six real-world temporal graphs show that DepTGL achieves an average speedup of 4.99x over state-of-the-art baselines, while maintaining comparable accuracy.
