Re-Think and Re-Design Graph Neural Networks in Spaces of Continuous Graph Diffusion Functionals

by   Tingting Dan, et al.

Graph neural networks (GNNs) are widely used in domains like social networks and biological systems. However, the locality assumption of GNNs, which limits information exchange to neighboring nodes, hampers their ability to capture long-range dependencies and global patterns in graphs. To address this, we propose a new inductive bias based on variational analysis, drawing inspiration from the Brachistochrone problem. Our framework establishes a mapping between discrete GNN models and continuous diffusion functionals. This enables the design of application-specific objective functions in the continuous domain and the construction of discrete deep models with mathematical guarantees. To tackle over-smoothing in GNNs, we analyze the existing layer-by-layer graph embedding models and identify that they are equivalent to l2-norm integral functionals of graph gradients, which cause over-smoothing. Similar to edge-preserving filters in image denoising, we introduce total variation (TV) to align the graph diffusion pattern with global community topologies. Additionally, we devise a selective mechanism to address the trade-off between model depth and over-smoothing, which can be easily integrated into existing GNNs. Furthermore, we propose a novel generative adversarial network (GAN) that predicts spreading flows in graphs through a neural transport equation. To mitigate vanishing flows, we customize the objective function to minimize transportation within each community while maximizing inter-community flows. Our GNN models achieve state-of-the-art (SOTA) performance on popular graph learning benchmarks such as Cora, Citeseer, and Pubmed.


page 15

page 16

page 21

page 23


Improving the Long-Range Performance of Gated Graph Neural Networks

Many popular variants of graph neural networks (GNNs) that are capable o...

Implicit Graph Neural Diffusion Based on Constrained Dirichlet Energy Minimization

Implicit graph neural networks (GNNs) have emerged as a potential approa...

ADR-GNN: Advection-Diffusion-Reaction Graph Neural Networks

Graph neural networks (GNNs) have shown remarkable success in learning r...

GRAND: Graph Neural Diffusion

We present Graph Neural Diffusion (GRAND) that approaches deep learning ...

Power to the Relational Inductive Bias: Graph Neural Networks in Electrical Power Grids

The application of graph neural networks (GNNs) to the domain of electri...

Neural Sheaf Diffusion: A Topological Perspective on Heterophily and Oversmoothing in GNNs

Cellular sheaves equip graphs with "geometrical" structure by assigning ...

Optimization-Induced Graph Implicit Nonlinear Diffusion

Due to the over-smoothing issue, most existing graph neural networks can...

Please sign up or login with your details

Forgot password? Click here to reset