DeepAI AI Chat
Log In Sign Up

Benchmarking Bayesian Deep Learning on Diabetic Retinopathy Detection Tasks

by   Neil Band, et al.

Bayesian deep learning seeks to equip deep neural networks with the ability to precisely quantify their predictive uncertainty, and has promised to make deep learning more reliable for safety-critical real-world applications. Yet, existing Bayesian deep learning methods fall short of this promise; new methods continue to be evaluated on unrealistic test beds that do not reflect the complexities of downstream real-world tasks that would benefit most from reliable uncertainty quantification. We propose the RETINA Benchmark, a set of real-world tasks that accurately reflect such complexities and are designed to assess the reliability of predictive models in safety-critical scenarios. Specifically, we curate two publicly available datasets of high-resolution human retina images exhibiting varying degrees of diabetic retinopathy, a medical condition that can lead to blindness, and use them to design a suite of automated diagnosis tasks that require reliable predictive uncertainty quantification. We use these tasks to benchmark well-established and state-of-the-art Bayesian deep learning methods on task-specific evaluation metrics. We provide an easy-to-use codebase for fast and easy benchmarking following reproducibility and software design principles. We provide implementations of all methods included in the benchmark as well as results computed over 100 TPU days, 20 GPU days, 400 hyperparameter configurations, and evaluation on at least 6 random seeds each.


page 2

page 20

page 21

page 22

page 23

page 24

page 36


A Comparison of Uncertainty Estimation Approaches in Deep Learning Components for Autonomous Vehicle Applications

A key to ensuring safety in Autonomous Vehicles (AVs) is to avoid any ab...

Towards Clear Expectations for Uncertainty Estimation

If Uncertainty Quantification (UQ) is crucial to achieve trustworthy Mac...

A Benchmark on Uncertainty Quantification for Deep Learning Prognostics

Reliable uncertainty quantification on RUL prediction is crucial for inf...

URSABench: Comprehensive Benchmarking of Approximate Bayesian Inference Methods for Deep Neural Networks

While deep learning methods continue to improve in predictive accuracy o...

Quantifying Uncertainty for Machine Learning Based Diagnostic

Virtual Diagnostic (VD) is a deep learning tool that can be used to pred...

A Systematic Comparison of Bayesian Deep Learning Robustness in Diabetic Retinopathy Tasks

Evaluation of Bayesian deep learning (BDL) methods is challenging. We of...

Benchmarking Probabilistic Deep Learning Methods for License Plate Recognition

Learning-based algorithms for automated license plate recognition implic...