Adaptive, Distribution-Free Prediction Intervals for Deep Neural Networks

by   Danijel Kivaranovic, et al.

This paper addresses the problem of assessing the variability of predictions from deep neural networks. There is a growing literature on using and improving the predictive accuracy of deep networks, but a concomitant improvement in the quantification of their uncertainty is lacking. We provide a prediction interval network (PI-Network) which is a transparent, tractable modification of the standard predictive loss used to train deep networks. The PI-Network outputs three values instead of a single point estimate and optimizes a loss function inspired by quantile regression. We go beyond merely motivating the construction of these networks and provide two prediction interval methods with provable, finite sample coverage guarantees without any assumptions on the underlying distribution from which our data is drawn. We only require that the observations are independent and identically distributed. Furthermore, our intervals adapt to heteroskedasticity and asymmetry in the conditional distribution of the response given the covariates. The first method leverages the conformal inference framework and provides average coverage. The second method provides a new, stronger guarantee by conditioning on the observed data. Lastly, our loss function does not compromise the predictive accuracy of the network like other prediction interval methods. We demonstrate the ease of use of the PI-Network as well as its improvements over other methods on both simulated and real data. As the PI-Network can be used with a host of deep learning methods with only minor modifications, its use should become standard practice, much like reporting standard errors along with mean estimates.


page 1

page 2

page 3

page 4


Training-conditional coverage for distribution-free predictive inference

The field of distribution-free predictive inference provides tools for p...

PI3NN: Prediction intervals from three independently trained neural networks

We propose a novel prediction interval method to learn prediction mean v...

Dual Accuracy-Quality-Driven Neural Network for Prediction Interval Generation

Accurate uncertainty quantification is necessary to enhance the reliabil...

Tight Prediction Intervals Using Expanded Interval Minimization

Prediction intervals are a valuable way of quantifying uncertainty in re...

With Malice Towards None: Assessing Uncertainty via Equalized Coverage

An important factor to guarantee a fair use of data-driven recommendatio...

Training Uncertainty-Aware Classifiers with Conformalized Deep Learning

Deep neural networks are powerful tools to detect hidden patterns in dat...

Exploiting Oxide Based Resistive RAM Variability for Probabilistic AI Hardware Design

Uncertainty plays a key role in real-time machine learning. As a signifi...

Please sign up or login with your details

Forgot password? Click here to reset