DGC: Training Dynamic Graphs with Spatio-Temporal Non-Uniformity using Graph Partitioning by Chunks

by   Fahao Chen, et al.

Dynamic Graph Neural Network (DGNN) has shown a strong capability of learning dynamic graphs by exploiting both spatial and temporal features. Although DGNN has recently received considerable attention by AI community and various DGNN models have been proposed, building a distributed system for efficient DGNN training is still challenging. It has been well recognized that how to partition the dynamic graph and assign workloads to multiple GPUs plays a critical role in training acceleration. Existing works partition a dynamic graph into snapshots or temporal sequences, which only work well when the graph has uniform spatio-temporal structures. However, dynamic graphs in practice are not uniformly structured, with some snapshots being very dense while others are sparse. To address this issue, we propose DGC, a distributed DGNN training system that achieves a 1.25x - 7.52x speedup over the state-of-the-art in our testbed. DGC's success stems from a new graph partitioning method that partitions dynamic graphs into chunks, which are essentially subgraphs with modest training workloads and few inter connections. This partitioning algorithm is based on graph coarsening, which can run very fast on large graphs. In addition, DGC has a highly efficient run-time, powered by the proposed chunk fusion and adaptive stale aggregation techniques. Extensive experimental results on 3 typical DGNN models and 4 popular dynamic graph datasets are presented to show the effectiveness of DGC.


page 3

page 4


ST-UNet: A Spatio-Temporal U-Network for Graph-structured Time Series Modeling

The spatio-temporal graph learning is becoming an increasingly important...

Unified Spatio-Temporal Modeling for Traffic Forecasting using Graph Neural Network

Research in deep learning models to forecast traffic intensities has gai...

Long-term Spatio-temporal Forecasting via Dynamic Multiple-Graph Attention

Many real-world ubiquitous applications, such as parking recommendations...

Partitioner Selection with EASE to Optimize Distributed Graph Processing

For distributed graph processing on massive graphs, a graph is partition...

Storing and Querying Large-Scale Spatio-Temporal Graphs with High-Throughput Edge Insertions

Real-world graphs often contain spatio-temporal information and evolve o...

Network Partitioning and Avoidable Contention

Network contention frequently dominates the run time of parallel algorit...

Dynamic Parallel and Distributed Graph Cuts

Graph-cuts are widely used in computer vision. In order to speed up the ...

Please sign up or login with your details

Forgot password? Click here to reset