Quantifying sources of uncertainty in drug discovery predictions with probabilistic models

by   Stanley E. Lazic, et al.

Knowing the uncertainty in a prediction is critical when making expensive investment decisions and when patient safety is paramount, but machine learning (ML) models in drug discovery typically provide only a single best estimate and ignore all sources of uncertainty. Predictions from these models may therefore be over-confident, which can put patients at risk and waste resources when compounds that are destined to fail are further developed. Probabilistic predictive models (PPMs) can incorporate uncertainty in both the data and model, and return a distribution of predicted values that represents the uncertainty in the prediction. PPMs not only let users know when predictions are uncertain, but the intuitive output from these models makes communicating risk easier and decision making better. Many popular machine learning methods have a PPM or Bayesian analogue, making PPMs easy to fit into current workflows. We use toxicity prediction as a running example, but the same principles apply for all prediction models used in drug discovery. The consequences of ignoring uncertainty and how PPMs account for uncertainty are also described. We aim to make the discussion accessible to a broad non-mathematical audience. Equations are provided to make ideas concrete for mathematical readers (but can be skipped without loss of understanding) and code is available for computational researchers (https://github.com/stanlazic/ML_uncertainty_quantification).


page 1

page 2

page 3

page 4


Identifying Bayesian Optimal Experiments for Uncertain Biochemical Pathway Models

Pharmacodynamic (PD) models are mathematical models of cellular reaction...

Bayesian Modelling in Practice: Using Uncertainty to Improve Trustworthiness in Medical Applications

The Intensive Care Unit (ICU) is a hospital department where machine lea...

Understanding Prediction Discrepancies in Machine Learning Classifiers

A multitude of classifiers can be trained on the same data to achieve si...

When Does Uncertainty Matter?: Understanding the Impact of Predictive Uncertainty in ML Assisted Decision Making

As machine learning (ML) models are increasingly being employed to assis...

A Meta-heuristic Approach to Estimate and Explain Classifier Uncertainty

Trust is a crucial factor affecting the adoption of machine learning (ML...

Bayesian Prediction for Artificial Intelligence

This paper shows that the common method used for making predictions unde...

Decision Tree Learning for Uncertain Clinical Measurements

Clinical decision requires reasoning in the presence of imperfect data. ...

Please sign up or login with your details

Forgot password? Click here to reset