DeepAI AI Chat
Log In Sign Up

DRew: Dynamically Rewired Message Passing with Delay

by   Benjamin Gutteridge, et al.

Message passing neural networks (MPNNs) have been shown to suffer from the phenomenon of over-squashing that causes poor performance for tasks relying on long-range interactions. This can be largely attributed to message passing only occurring locally, over a node's immediate neighbours. Rewiring approaches attempting to make graphs 'more connected', and supposedly better suited to long-range tasks, often lose the inductive bias provided by distance on the graph since they make distant nodes communicate instantly at every layer. In this paper we propose a framework, applicable to any MPNN architecture, that performs a layer-dependent rewiring to ensure gradual densification of the graph. We also propose a delay mechanism that permits skip connections between nodes depending on the layer and their mutual distance. We validate our approach on several long-range tasks and show that it outperforms graph Transformers and multi-hop MPNNs.


page 1

page 2

page 3

page 4


Dynamic Graph Message Passing Networks

Modelling long-range dependencies is critical for complex scene understa...

Ewald-based Long-Range Message Passing for Molecular Graphs

Neural architectures that learn potential energy surfaces from molecular...

Anti-Symmetric DGN: a stable architecture for Deep Graph Networks

Deep Graph Networks (DGNs) currently dominate the research landscape of ...

Dynamic Graph Message Passing Networks for Visual Recognition

Modelling long-range dependencies is critical for scene understanding ta...

Where Did the Gap Go? Reassessing the Long-Range Graph Benchmark

The recent Long-Range Graph Benchmark (LRGB, Dwivedi et al. 2022) introd...

Dimensionality Reduction Meets Message Passing for Graph Node Embeddings

Graph Neural Networks (GNNs) have become a popular approach for various ...

PushNet: Efficient and Adaptive Neural Message Passing

Message passing neural networks have recently evolved into a state-of-th...