Joint Embedding of Structural and Functional Brain Networks with Graph Neural Networks for Mental Illness Diagnosis

by   Yanqiao Zhu, et al.

Multimodal brain networks characterize complex connectivities among different brain regions from both structural and functional aspects and provide a new means for mental disease analysis. Recently, Graph Neural Networks (GNNs) have become a de facto model for analyzing graph-structured data. However, how to employ GNNs to extract effective representations from brain networks in multiple modalities remains rarely explored. Moreover, as brain networks provide no initial node features, how to design informative node attributes and leverage edge weights for GNNs to learn is left unsolved. To this end, we develop a novel multiview GNN for multimodal brain networks. In particular, we regard each modality as a view for brain networks and employ contrastive learning for multimodal fusion. Then, we propose a GNN model which takes advantage of the message passing scheme by propagating messages based on degree statistics and brain region connectivities. Extensive experiments on two real-world disease datasets (HIV and Bipolar) demonstrate the effectiveness of our proposed method over state-of-the-art baselines.


BrainGB: A Benchmark for Brain Network Analysis with Graph Neural Networks

Mapping the connectome of the human brain using structural or functional...

Deep Reinforcement Learning Guided Graph Neural Networks for Brain Network Analysis

Modern neuroimaging techniques, such as diffusion tensor imaging (DTI) a...

Single-Cell Multimodal Prediction via Transformers

The recent development of multimodal single-cell technology has made the...

Deep Representation Learning For Multimodal Brain Networks

Applying network science approaches to investigate the functions and ana...

How Expressive are Graph Neural Networks in Recommendation?

Graph Neural Networks (GNNs) have demonstrated superior performance on v...

Benchmarking Graph Neural Networks for FMRI analysis

Graph Neural Networks (GNNs) have emerged as a powerful tool to learn fr...

Tackling Oversmoothing of GNNs with Contrastive Learning

Graph neural networks (GNNs) integrate the comprehensive relation of gra...

Please sign up or login with your details

Forgot password? Click here to reset