Robust Learning via Ensemble Density Propagation in Deep Neural Networks

by   Giuseppina Carannante, et al.
The University of Sheffield
Rowan University

Learning in uncertain, noisy, or adversarial environments is a challenging task for deep neural networks (DNNs). We propose a new theoretically grounded and efficient approach for robust learning that builds upon Bayesian estimation and Variational Inference. We formulate the problem of density propagation through layers of a DNN and solve it using an Ensemble Density Propagation (EnDP) scheme. The EnDP approach allows us to propagate moments of the variational probability distribution across the layers of a Bayesian DNN, enabling the estimation of the mean and covariance of the predictive distribution at the output of the model. Our experiments using MNIST and CIFAR-10 datasets show a significant improvement in the robustness of the trained models to random noise and adversarial attacks.


page 1

page 2

page 3

page 4


EMPIR: Ensembles of Mixed Precision Deep Networks for Increased Robustness against Adversarial Attacks

Ensuring robustness of Deep Neural Networks (DNNs) is crucial to their a...

Density Uncertainty Layers for Reliable Uncertainty Estimation

Assessing the predictive uncertainty of deep neural networks is crucial ...

Stabilising priors for robust Bayesian deep learning

Bayesian neural networks (BNNs) have developed into useful tools for pro...

VarMixup: Exploiting the Latent Space for Robust Training and Inference

The vulnerability of Deep Neural Networks (DNNs) to adversarial attacks ...

Adv-BNN: Improved Adversarial Defense through Robust Bayesian Neural Network

We present a new algorithm to train a robust neural network against adve...

CARE: Certifiably Robust Learning with Reasoning via Variational Inference

Despite great recent advances achieved by deep neural networks (DNNs), t...

Please sign up or login with your details

Forgot password? Click here to reset