Practical Posterior Error Bounds from Variational Objectives

10/09/2019
by   Jonathan H. Huggins, et al.
23

Variational inference has become an increasingly attractive, computationally efficient alternative to Markov chain Monte Carlo methods for approximate Bayesian inference. However, a major obstacle to the widespread use of variational methods is the lack of accuracy measures for variational approximations that are both theoretically justified and computationally efficient. In this paper, we provide rigorous bounds on the error of posterior mean and uncertainty estimates that arise from full-distribution approximations, as in variational inference. Our bounds are widely applicable as they require only that the approximating and exact posteriors have polynomial moments. Our bounds are computationally efficient for variational inference in that they require only standard values from variational objectives, straightforward analytic calculations, and simple Monte Carlo estimates. We show that our analysis naturally leads to a new and improved workflow for variational inference. Finally, we demonstrate the utility of our proposed workflow and error bounds on a real-data example with a widely used multilevel hierarchical model.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
09/25/2018

Practical bounds on the error of Bayesian posterior approximations: A nonasymptotic approach

Bayesian inference typically requires the computation of an approximatio...
research
10/09/2018

Fixing Variational Bayes: Deterministic Variational Inference for Bayesian Neural Networks

Bayesian neural networks (BNNs) hold great promise as a flexible and pri...
research
07/04/2015

Inference for determinantal point processes without spectral knowledge

Determinantal point processes (DPPs) are point process models that natur...
research
11/01/2016

Variational Inference via χ-Upper Bound Minimization

Variational inference (VI) is widely used as an efficient alternative to...
research
02/24/2023

A Targeted Accuracy Diagnostic for Variational Approximations

Variational Inference (VI) is an attractive alternative to Markov Chain ...
research
03/08/2022

Variational Inference with Locally Enhanced Bounds for Hierarchical Models

Hierarchical models represent a challenging setting for inference algori...
research
10/10/2019

Rate Optimal Variational Bayesian Inference for Sparse DNN

Sparse deep neural network (DNN) has drawn much attention in recent stud...

Please sign up or login with your details

Forgot password? Click here to reset