Bayesian Hypernetworks

by   David Krueger, et al.
Element AI Inc
Université de Montréal
McGill University

We propose Bayesian hypernetworks: a framework for approximate Bayesian inference in neural networks. A Bayesian hypernetwork, h, is a neural network which learns to transform a simple noise distribution, p(ϵ) = N(0,I), to a distribution q(θ) q(h(ϵ)) over the parameters θ of another neural network (the "primary network"). We train q with variational inference, using an invertible h to enable efficient estimation of the variational lower bound on the posterior p(θ | D) via sampling. In contrast to most methods for Bayesian deep learning, Bayesian hypernets can represent a complex multimodal approximate posterior with correlations between parameters, while enabling cheap i.i.d. sampling of q(θ). We demonstrate these qualitative advantages of Bayesian hypernets, which also achieve competitive performance on a suite of tasks that demonstrate the advantage of estimating model uncertainty, including active learning and anomaly detection.


page 1

page 2

page 3

page 4


Stochastic Bayesian Neural Networks

Bayesian neural networks perform variational inference over weights but ...

Efficient variational Bayesian neural network ensembles for outlier detection

In this work we perform outlier detection using ensembles of neural netw...

Adversarial Distillation of Bayesian Neural Network Posteriors

Bayesian neural networks (BNNs) allow us to reason about uncertainty in ...

Stochastic Neural Network with Kronecker Flow

Recent advances in variational inference enable the modelling of highly ...

Using Deep Neural Network Approximate Bayesian Network

We present a new method to approximate posterior probabilities of Bayesi...

A Generative Deep Recurrent Model for Exchangeable Data

We present a novel model architecture which leverages deep learning tool...

Sampling-free Variational Inference for Neural Networks with Multiplicative Activation Noise

To adopt neural networks in safety critical domains, knowing whether we ...

Please sign up or login with your details

Forgot password? Click here to reset