Transfer Learning of Graph Neural Networks with Ego-graph Information Maximization

by   Qi Zhu, et al.

Graph neural networks (GNNs) have been shown with superior performance in various applications, but training dedicated GNNs can be costly for large-scale graphs. Some recent work started to study the pre-training of GNNs. However, none of them provide theoretical insights into the design of their frameworks, or clear requirements and guarantees towards the transferability of GNNs. In this work, we establish a theoretically grounded and practically useful framework for the transfer learning of GNNs. Firstly, we propose a novel view towards the essential graph information and advocate the capturing of it as the goal of transferable GNN training, which motivates the design of Ours, a novel GNN framework based on ego-graph information maximization to analytically achieve this goal. Secondly, we specify the requirement of structure-respecting node features as the GNN input, and derive a rigorous bound of GNN transferability based on the difference between the local graph Laplacians of the source and target graphs. Finally, we conduct controlled synthetic experiments to directly justify our theoretical conclusions. Extensive experiments on real-world networks towards role identification show consistent results in the rigorously analyzed setting of direct-transfering, while those towards large-scale relation prediction show promising results in the more generalized and practical setting of transfering with fine-tuning.


page 1

page 2

page 3

page 4


Transferability of Graph Neural Networks using Graphon and Sampling Theories

Graph neural networks (GNNs) have become powerful tools for processing g...

Investigating Transfer Learning in Graph Neural Networks

Graph neural networks (GNNs) build on the success of deep learning model...

GSN: A Universal Graph Neural Network Inspired by Spring Network

The design of universal Graph Neural Networks (GNNs) that operate on bot...

DiP-GNN: Discriminative Pre-Training of Graph Neural Networks

Graph neural network (GNN) pre-training methods have been proposed to en...

Snowflake: Scaling GNNs to High-Dimensional Continuous Control via Parameter Freezing

Recent research has shown that Graph Neural Networks (GNNs) can learn po...

Program-to-Circuit: Exploiting GNNs for Program Representation and Circuit Translation

Circuit design is complicated and requires extensive domain-specific exp...

Graph Neural Networks with Heterophily

Graph Neural Networks (GNNs) have proven to be useful for many different...

Please sign up or login with your details

Forgot password? Click here to reset