On the Importance of Strong Baselines in Bayesian Deep Learning

by   Jishnu Mukhoti, et al.
University of Oxford

Like all sub-fields of machine learning, Bayesian Deep Learning is driven by empirical validation of its theoretical proposals. Given the many aspects of an experiment, it is always possible that minor or even major experimental flaws can slip by both authors and reviewers. One of the most popular experiments used to evaluate approximate inference techniques is the regression experiment on UCI datasets. However, in this experiment, models which have been trained to convergence have often been compared with baselines trained only for a fixed number of iterations. What we find is that if we take a well-established baseline and evaluate it under the same experimental settings, it shows significant improvements in performance. In fact, it outperforms or performs competitively with numerous to several methods that when they were introduced claimed to be superior to the very same baseline method. Hence, by exposing this flaw in experimental procedure, we highlight the importance of using identical experimental setups to evaluate, compare and benchmark methods in Bayesian Deep Learning.


page 1

page 2

page 3

page 4


Hybridizing Physical and Data-driven Prediction Methods for Physicochemical Properties

We present a generic way to hybridize physical and data-driven methods f...

Priors in Bayesian Deep Learning: A Review

While the choice of prior is one of the most critical parts of the Bayes...

AdaSelection: Accelerating Deep Learning Training through Data Subsampling

In this paper, we introduce AdaSelection, an adaptive sub-sampling metho...

Bayesian LSTMs in medicine

The medical field stands to see significant benefits from the recent adv...

Uncertainty Baselines: Benchmarks for Uncertainty Robustness in Deep Learning

High-quality estimates of uncertainty and robustness are crucial for num...

Cranial Implant Design via Virtual Craniectomy with Shape Priors

Cranial implant design is a challenging task, whose accuracy is crucial ...

DeepCore: A Comprehensive Library for Coreset Selection in Deep Learning

Coreset selection, which aims to select a subset of the most informative...

Please sign up or login with your details

Forgot password? Click here to reset