Why So Many Published Sensitivity Analyses Are False. A Systematic Review of Sensitivity Analysis Practices

by   Andrea Saltelli, et al.

Sensitivity analysis (SA) has much to offer for a very large class of applications, such as model selection, calibration, optimization, quality assurance and many others. Sensitivity analysis offers crucial contextual information regarding a prediction by answering the question "Which uncertain input factors are responsible for the uncertainty in the prediction?" SA is distinct from uncertainty analysis (UA), which instead addresses the question "How uncertain is the prediction?" As we discuss in the present paper much confusion exists in the use of these terms. A proper uncertainty analysis of the output of a mathematical model needs to map what the model does when the input factors are left free to vary over their range of existence. A fortiori, this is true of a sensitivity analysis. Despite this, most UA and SA still explore the input space; moving along mono-dimensional corridors which leave the space of variation of the input factors mostly unscathed. We use results from a bibliometric analysis to show that many published SA fail the elementary requirement to properly explore the space of the input factors. The results, while discipline-dependent, point to a worrying lack of standards and of recognized good practices. The misuse of sensitivity analysis in mathematical modelling is at least as serious as the misuse of the p-test in statistical modelling. Mature methods have existed for about two decades to produce a defensible sensitivity analysis. We end by offering a rough guide for proper use of the methods.


page 20

page 21

page 25

page 26


Uncertainty and Sensitivity Analyses Methods for Agent-Based Mathematical Models: An Introductory Review

Multiscale, agent-based mathematical models of biological systems are of...

Variance-based sensitivity analysis: The quest for better estimators and designs between explorativity and economy

Variance-based sensitivity indices have established themselves as a refe...

Does modelling need a Reformation? Ideas for a new grammar of modelling

The quality of mathematical modelling is looked at from the perspective ...

An information upper bound for probability sensitivity

Uncertain input of a mathematical model induces uncertainties in the out...

Sensitivity analysis methods in the biomedical sciences

Sensitivity analysis is an important part of a mathematical modeller's t...

A sensitivity analysis of the PAWN sensitivity index

The PAWN index is gaining traction among the modelling community as a mo...

Target and Conditional Sensitivity Analysis with Emphasis on Dependence Measures

In the context of sensitivity analysis of complex phenomena in presence ...

Please sign up or login with your details

Forgot password? Click here to reset