Analyzing the Performance of Graph Neural Networks with Pipe Parallelism

12/20/2020
by   Matthew T. Dearing, et al.
0

Many interesting datasets ubiquitous in machine learning and deep learning can be described via graphs. As the scale and complexity of graph-structured datasets increase, such as in expansive social networks, protein folding, chemical interaction networks, and material phase transitions, improving the efficiency of the machine learning techniques applied to these is crucial. In this study, we focus on Graph Neural Networks (GNN), which have found great success in tasks such as node or edge classification and link prediction. However, standard GNN models have scaling limits due to necessary recursive calculations performed through dense graph relationships that lead to memory and runtime bottlenecks. While new approaches for processing larger networks are needed to advance graph techniques, and several have been proposed, we study how GNNs could be parallelized using existing tools and frameworks that are already known to be successful in the deep learning community. In particular, we investigate applying pipeline parallelism to GNN models with GPipe, introduced by Google in 2018.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/19/2022

Parallel and Distributed Graph Neural Networks: An In-Depth Concurrency Analysis

Graph neural networks (GNNs) are among the most powerful tools in deep l...
research
10/04/2021

Scaling Graph-based Deep Learning models to larger networks

Graph Neural Networks (GNN) have shown a strong potential to be integrat...
research
11/23/2021

Network In Graph Neural Network

Graph Neural Networks (GNNs) have shown success in learning from graph s...
research
02/27/2020

Bridging the Gap between Spatial and Spectral Domains: A Survey on Graph Neural Networks

The success of deep learning has been widely recognized in many machine ...
research
10/31/2022

GNN at the Edge: Cost-Efficient Graph Neural Network Processing over Distributed Edge Servers

Edge intelligence has arisen as a promising computing paradigm for suppo...
research
07/21/2021

Bridging the Gap between Spatial and Spectral Domains: A Theoretical Framework for Graph Neural Networks

During the past decade, deep learning's performance has been widely reco...
research
05/10/2023

Graph Neural Networks and 3-Dimensional Topology

We test the efficiency of applying Geometric Deep Learning to the proble...

Please sign up or login with your details

Forgot password? Click here to reset