AGNN: Alternating Graph-Regularized Neural Networks to Alleviate Over-Smoothing

by   Zhaoliang Chen, et al.

Graph Convolutional Network (GCN) with the powerful capacity to explore graph-structural data has gained noticeable success in recent years. Nonetheless, most of the existing GCN-based models suffer from the notorious over-smoothing issue, owing to which shallow networks are extensively adopted. This may be problematic for complex graph datasets because a deeper GCN should be beneficial to propagating information across remote neighbors. Recent works have devoted effort to addressing over-smoothing problems, including establishing residual connection structure or fusing predictions from multi-layer models. Because of the indistinguishable embeddings from deep layers, it is reasonable to generate more reliable predictions before conducting the combination of outputs from various layers. In light of this, we propose an Alternating Graph-regularized Neural Network (AGNN) composed of Graph Convolutional Layer (GCL) and Graph Embedding Layer (GEL). GEL is derived from the graph-regularized optimization containing Laplacian embedding term, which can alleviate the over-smoothing problem by periodic projection from the low-order feature space onto the high-order space. With more distinguishable features of distinct layers, an improved Adaboost strategy is utilized to aggregate outputs from each layer, which explores integrated embeddings of multi-hop neighbors. The proposed model is evaluated via a large number of experiments including performance comparison with some multi-layer or multi-order graph neural networks, which reveals the superior performance improvement of AGNN compared with state-of-the-art models.


page 1

page 4

page 11


Layer-refined Graph Convolutional Networks for Recommendation

Recommendation models utilizing Graph Convolutional Networks (GCNs) have...

PathSAGE: Spatial Graph Attention Neural Networks With Random Path Sampling

Graph Convolutional Networks (GCNs) achieve great success in non-Euclide...

Multi-hop Graph Convolutional Network with High-order Chebyshev Approximation for Text Reasoning

Graph convolutional network (GCN) has become popular in various natural ...

Interest-aware Message-Passing GCN for Recommendation

Graph Convolution Networks (GCNs) manifest great potential in recommenda...

Tree Decomposed Graph Neural Network

Graph Neural Networks (GNNs) have achieved significant success in learni...

Stability and Generalization of ℓ_p-Regularized Stochastic Learning for GCN

Graph convolutional networks (GCN) are viewed as one of the most popular...

Recursive Graphical Neural Networks for Text Classification

The complicated syntax structure of natural language is hard to be expli...

Please sign up or login with your details

Forgot password? Click here to reset