DropMessage: Unifying Random Dropping for Graph Neural Networks

by   Taoran Fang, et al.

Graph Neural Networks (GNNs) are powerful tools for graph representation learning. Despite their rapid development, GNNs also faces some challenges, such as over-fitting, over-smoothing, and non-robustness. Previous works indicate that these problems can be alleviated by random dropping methods, which integrate noises into models by randomly masking parts of the input. However, some open-ended problems of random dropping on GNNs remain to solve. First, it is challenging to find a universal method that are suitable for all cases considering the divergence of different datasets and models. Second, random noises introduced to GNNs cause the incomplete coverage of parameters and unstable training process. In this paper, we propose a novel random dropping method called DropMessage, which performs dropping operations directly on the message matrix and can be applied to any message-passing GNNs. Furthermore, we elaborate the superiority of DropMessage: it stabilizes the training process by reducing sample variance; it keeps information diversity from the perspective of information theory, which makes it a theoretical upper bound of other methods. Also, we unify existing random dropping methods into our framework and analyze their effects on GNNs. To evaluate our proposed method, we conduct experiments that aims for multiple tasks on five public datasets and two industrial datasets with various backbone models. The experimental results show that DropMessage has both advantages of effectiveness and generalization.


page 1

page 2

page 3

page 4


Unifying gradient regularization for Heterogeneous Graph Neural Networks

Heterogeneous Graph Neural Networks (HGNNs) are a class of powerful deep...

EEGNN: Edge Enhanced Graph Neural Networks

Training deep graph neural networks (GNNs) poses a challenging task, as ...

A Comparative Study on Robust Graph Neural Networks to Structural Noises

Graph neural networks (GNNs) learn node representations by passing and a...

Simulating Liquids with Graph Networks

Simulating complex dynamics like fluids with traditional simulators is c...

Interpreting and Unifying Graph Neural Networks with An Optimization Framework

Graph Neural Networks (GNNs) have received considerable attention on gra...

GraphGLOW: Universal and Generalizable Structure Learning for Graph Neural Networks

Graph structure learning is a well-established problem that aims at opti...

Distribution shift mitigation at test time with performance guarantees

Due to inappropriate sample selection and limited training data, a distr...

Please sign up or login with your details

Forgot password? Click here to reset