Excess risk analysis for epistemic uncertainty with application to variational inference

06/02/2022
by   Futoshi Futami, et al.
0

We analyze the epistemic uncertainty (EU) of supervised learning in Bayesian inference by focusing on the excess risk. Existing analysis is limited to the Bayesian setting, which assumes a correct model and exact Bayesian posterior distribution. Thus we cannot apply the existing theory to modern Bayesian algorithms, such as variational inference. To address this, we present a novel EU analysis in the frequentist setting, where data is generated from an unknown distribution. We show a relation between the generalization ability and the widely used EU measurements, such as the variance and entropy of the predictive distribution. Then we show their convergence behaviors theoretically. Finally, we propose new variational inference that directly controls the prediction and EU evaluation performances based on the PAC-Bayesian theory. Numerical experiments show that our algorithm significantly improves the EU evaluation over the existing methods.

READ FULL TEXT

page 1

page 2

page 3

page 4

06/09/2021

Loss function based second-order Jensen inequality and its application to particle variational inference

Bayesian model averaging, obtained as the expectation of a likelihood fu...
04/08/2019

A Generalization Bound for Online Variational Inference

Bayesian inference provides an attractive online-learning framework to a...
02/25/2020

Variational Inference and Bayesian CNNs for Uncertainty Estimation in Multi-Factorial Bone Age Prediction

Additionally to the extensive use in clinical medicine, biological age (...
05/27/2020

VarFA: A Variational Factor Analysis Framework For Efficient Bayesian Learning Analytics

We propose VarFA, a variational inference factor analysis framework that...
10/24/2020

Implicit Variational Inference: the Parameter and the Predictor Space

Having access to accurate confidence levels along with the predictions a...
05/09/2023

Fully Bayesian VIB-DeepSSM

Statistical shape modeling (SSM) enables population-based quantitative a...
06/10/2019

Stochastic Neural Network with Kronecker Flow

Recent advances in variational inference enable the modelling of highly ...

Please sign up or login with your details

Forgot password? Click here to reset