BenchTemp: A General Benchmark for Evaluating Temporal Graph Neural Networks

by   Qiang Huang, et al.

To handle graphs in which features or connectivities are evolving over time, a series of temporal graph neural networks (TGNNs) have been proposed. Despite the success of these TGNNs, the previous TGNN evaluations reveal several limitations regarding four critical issues: 1) inconsistent datasets, 2) inconsistent evaluation pipelines, 3) lacking workload diversity, and 4) lacking efficient comparison. Overall, there lacks an empirical study that puts TGNN models onto the same ground and compares them comprehensively. To this end, we propose BenchTemp, a general benchmark for evaluating TGNN models on various workloads. BenchTemp provides a set of benchmark datasets so that different TGNN models can be fairly compared. Further, BenchTemp engineers a standard pipeline that unifies the TGNN evaluation. With BenchTemp, we extensively compare the representative TGNN models on different tasks (e.g., link prediction and node classification) and settings (transductive and inductive), w.r.t. both effectiveness and efficiency metrics. We have made BenchTemp publicly available at


page 13

page 25


Wiki-CS: A Wikipedia-Based Benchmark for Graph Neural Networks

We present Wiki-CS, a novel dataset derived from Wikipedia for benchmark...

BAGEL: A Benchmark for Assessing Graph Neural Network Explanations

The problem of interpreting the decisions of machine learning is a well-...

An Empirical Evaluation of Temporal Graph Benchmark

In this paper, we conduct an empirical evaluation of Temporal Graph Benc...

Towards Better Dynamic Graph Learning: New Architecture and Unified Library

We propose DyGFormer, a new Transformer-based architecture for dynamic g...

Temporal Graph Benchmark for Machine Learning on Temporal Graphs

We present the Temporal Graph Benchmark (TGB), a collection of challengi...

Distribution Free Prediction Sets for Node Classification

Graph Neural Networks (GNNs) are able to achieve high classification acc...

Evaluating Graph Neural Networks for Link Prediction: Current Pitfalls and New Benchmarking

Link prediction attempts to predict whether an unseen edge exists based ...

Please sign up or login with your details

Forgot password? Click here to reset