gSuite: A Flexible and Framework Independent Benchmark Suite for Graph Neural Network Inference on GPUs

by   Taha Tekdogan, et al.

As the interest to Graph Neural Networks (GNNs) is growing, the importance of benchmarking and performance characterization studies of GNNs is increasing. So far, we have seen many studies that investigate and present the performance and computational efficiency of GNNs. However, the work done so far has been carried out using a few high-level GNN frameworks. Although these frameworks provide ease of use, they contain too many dependencies to other existing libraries. The layers of implementation details and the dependencies complicate the performance analysis of GNN models that are built on top of these frameworks, especially while using architectural simulators. Furthermore, different approaches on GNN computation are generally overlooked in prior characterization studies, and merely one of the common computational models is evaluated. Based on these shortcomings and needs that we observed, we developed a benchmark suite that is framework independent, supporting versatile computational models, easily configurable and can be used with architectural simulators without additional effort. Our benchmark suite, which we call gSuite, makes use of only hardware vendor's libraries and therefore it is independent of any other frameworks. gSuite enables performing detailed performance characterization studies on GNN Inference using both contemporary GPU profilers and architectural GPU simulators. To illustrate the benefits of our new benchmark suite, we perform a detailed characterization study with a set of well-known GNN models with various datasets; running gSuite both on a real GPU card and a timing-detailed GPU simulator. We also implicate the effect of computational models on performance. We use several evaluation metrics to rigorously measure the performance of GNN computation.


page 1

page 6

page 8

page 9


Characterizing the Efficiency of Graph Neural Network Frameworks with a Magnifying Glass

Graph neural networks (GNNs) have received great attention due to their ...

Architectural Implications of Graph Neural Networks

Graph neural networks (GNN) represent an emerging line of deep learning ...

Beyond Graph Neural Networks with Lifted Relational Neural Networks

We demonstrate a declarative differentiable programming framework based ...

Design Space for Graph Neural Networks

The rapid evolution of Graph Neural Networks (GNNs) has led to a growing...

GraphWorld: Fake Graphs Bring Real Insights for GNNs

Despite advances in the field of Graph Neural Networks (GNNs), only a sm...

Empowering GNNs with Fine-grained Communication-Computation Pipelining on Multi-GPU Platforms

The increasing size of input graphs for graph neural networks (GNNs) hig...

ENIGMA Anonymous: Symbol-Independent Inference Guiding Machine (system description)

We describe an implementation of gradient boosting and neural guidance o...

Please sign up or login with your details

Forgot password? Click here to reset