Learned Low Precision Graph Neural Networks

by   Yiren Zhao, et al.

Deep Graph Neural Networks (GNNs) show promising performance on a range of graph tasks, yet at present are costly to run and lack many of the optimisations applied to DNNs. We show, for the first time, how to systematically quantise GNNs with minimal or no loss in performance using Network Architecture Search (NAS). We define the possible quantisation search space of GNNs. The proposed novel NAS mechanism, named Low Precision Graph NAS (LPGNAS), constrains both architecture and quantisation choices to be differentiable. LPGNAS learns the optimal architecture coupled with the best quantisation strategy for different components in the GNN automatically using back-propagation in a single search round. On eight different datasets, solving the task of classifying unseen nodes in a graph, LPGNAS generates quantised models with significant reductions in both model and buffer sizes but with similar accuracy to manually designed networks and other NAS results. In particular, on the Pubmed dataset, LPGNAS shows a better size-accuracy Pareto frontier compared to seven other manual and searched baselines, offering a 2.3 times reduction in model size but a 0.4 the best NAS competitor. Finally, from our collected quantisation statistics on a wide range of datasets, we suggest a W4A8 (4-bit weights, 8-bit activations) quantisation strategy might be the bottleneck for naive GNN quantisations.


Probabilistic Dual Network Architecture Search on Graphs

We present the first differentiable Network Architecture Search (NAS) fo...

Adversarially Robust Neural Architecture Search for Graph Neural Networks

Graph Neural Networks (GNNs) obtain tremendous success in modeling relat...

Efficient and Explainable Graph Neural Architecture Search via Monte-Carlo Tree Search

Graph neural networks (GNNs) are powerful tools for performing data scie...

Auto-HeG: Automated Graph Neural Network on Heterophilic Graphs

Graph neural architecture search (NAS) has gained popularity in automati...

Graph Neural Network Architecture Search for Molecular Property Prediction

Predicting the properties of a molecule from its structure is a challeng...

Graph Element Networks: adaptive, structured computation and memory

We explore the use of graph neural networks (GNNs) to model spatial proc...

Search What You Want: Barrier Panelty NAS for Mixed Precision Quantization

Emergent hardwares can support mixed precision CNN models inference that...

Please sign up or login with your details

Forgot password? Click here to reset