A comprehensive study on the prediction reliability of graph neural networks for virtual screening

by   Soojung Yang, et al.

Prediction models based on deep neural networks are increasingly gaining attention for fast and accurate virtual screening systems. For decision makings in virtual screening, researchers find it useful to interpret an output of classification system as probability, since such interpretation allows them to filter out more desirable compounds. However, probabilistic interpretation cannot be correct for models that hold over-parameterization problems or inappropriate regularizations, leading to unreliable prediction and decision making. In this regard, we concern the reliability of neural prediction models on molecular properties, especially when models are trained with sparse data points and imbalanced distributions. This work aims to propose guidelines for training reliable models, we thus provide methodological details and ablation studies on the following train principles. We investigate the effects of model architectures, regularization methods, and loss functions on the prediction performance and reliability of classification results. Moreover, we evaluate prediction reliability of models on virtual screening scenario. Our result highlights that correct choice of regularization and inference methods is evidently important to achieve high success rate, especially in data imbalanced situation. All experiments were performed under a single unified model implementation to alleviate external randomness in model training and to enable precise comparison of results.


page 15

page 17

page 19


A benchmark study on reliable molecular supervised learning via Bayesian learning

Virtual screening aims to find desirable compounds from chemical library...

Regression Enrichment Surfaces: a Simple Analysis Technique for Virtual Drug Screening Models

We present a new method for understanding the performance of a model in ...

The Impact of Class Rebalancing Techniques on the Performance and Interpretation of Defect Prediction Models

Defect prediction models that are trained on class imbalanced datasets (...

Compound virtual screening by learning-to-rank with gradient boosting decision tree and enrichment-based cumulative gain

Learning-to-rank, a machine learning technique widely used in informatio...

Silas: High Performance, Explainable and Verifiable Machine Learning

This paper introduces a new classification tool named Silas, which is bu...

Scenario-based Evaluation of Prediction Models for Automated Vehicles

To operate safely, an automated vehicle (AV) must anticipate how the env...

A performance characteristic curve for model evaluation: the application in information diffusion prediction

The information diffusion prediction on social networks aims to predict ...

Please sign up or login with your details

Forgot password? Click here to reset