Fast Bayesian Updates for Deep Learning with a Use Case in Active Learning

by   Marek Herde, et al.

Retraining deep neural networks when new data arrives is typically computationally expensive. Moreover, certain applications do not allow such costly retraining due to time or computational constraints. Fast Bayesian updates are a possible solution to this issue. Therefore, we propose a Bayesian update based on Monte-Carlo samples and a last-layer Laplace approximation for different Bayesian neural network types, i.e., Dropout, Ensemble, and Spectral Normalized Neural Gaussian Process (SNGP). In a large-scale evaluation study, we show that our updates combined with SNGP represent a fast and competitive alternative to costly retraining. As a use case, we combine the Bayesian updates for SNGP with different sequential query strategies to exemplarily demonstrate their improved selection performance in active learning.


Episode-Based Active Learning with Bayesian Neural Networks

We investigate different strategies for active learning with Bayesian de...

Efficacy of Bayesian Neural Networks in Active Learning

Obtaining labeled data for machine learning tasks can be prohibitively e...

Deep Ensemble Bayesian Active Learning : Addressing the Mode Collapse issue in Monte Carlo dropout via Ensembles

In image classification tasks, the ability of deep CNNs to deal with com...

Sequential Experimental Design for Spectral Measurement: Active Learning Using a Parametric Model

In this study, we demonstrate a sequential experimental design for spect...

Active-learning-based non-intrusive Model Order Reduction

The Model Order Reduction (MOR) technique can provide compact numerical ...

Active Learning for Sequence Tagging with Deep Pre-trained Models and Bayesian Uncertainty Estimates

Annotating training data for sequence tagging tasks is usually very time...

Actively learning a Bayesian matrix fusion model with deep side information

High-dimensional deep neural network representations of images and conce...

Please sign up or login with your details

Forgot password? Click here to reset