Optimizing Graph Transformer Networks with Graph-based Techniques

by   Loc Hoang, et al.

Graph transformer networks (GTN) are a variant of graph convolutional networks (GCN) that are targeted to heterogeneous graphs in which nodes and edges have associated type information that can be exploited to improve inference accuracy. GTNs learn important metapaths in the graph, create weighted edges for these metapaths, and use the resulting graph in a GCN. Currently, the only available implementation of GTNs uses dense matrix multiplication to find metapaths. Unfortunately, the space overhead of this approach can be large, so in practice it is used only for small graphs. In addition, the matrix-based implementation is not fine-grained enough to use random-walk based methods to optimize metapath finding. In this paper, we present a graph-based formulation and implementation of the GTN metapath finding problem. This graph-based formulation has two advantages over the matrix-based approach. First, it is more space efficient than the original GTN implementation and more compute-efficient for metapath sizes of practical interest. Second, it permits us to implement a sampling method that reduces the number of metapaths that must be enumerated, allowing the implementation to be used for larger graphs and larger metapath sizes. Experimental results show that our implementation is 6.5× faster than the original GTN implementation on average for a metapath length of 4, and our sampling implementation is 155× faster on average than this implementation without compromising on the accuracy of the GTN.


Generalized Multi-Relational Graph Convolution Network

Graph Convolutional Networks (GCNs) have received increasing attention i...

Pseudoinverse Graph Convolutional Networks: Fast Filters Tailored for Large Eigengaps of Dense Graphs and Hypergraphs

Graph Convolutional Networks (GCNs) have proven to be successful tools f...

GraphSAINT: Graph Sampling Based Inductive Learning Method

Graph Convolutional Networks (GCNs) are powerful models for learning rep...

Confidence-based Graph Convolutional Networks for Semi-Supervised Learning

Predicting properties of nodes in a graph is an important problem with a...

Scaling betweenness centrality using communication-efficient sparse matrix multiplication

Betweenness centrality (BC) is a crucial graph problem that measures the...

Solving Graph Laplacians via Multilevel Sparsifiers

We consider effective preconditioners for solving Laplacians of general ...

Uncertainty-Based Dynamic Graph Neighborhoods For Medical Segmentation

In recent years, deep learning based methods have shown success in essen...

Please sign up or login with your details

Forgot password? Click here to reset