Boosting Variational Inference

11/17/2016
by   Fangjian Guo, et al.
0

Variational inference (VI) provides fast approximations of a Bayesian posterior in part because it formulates posterior approximation as an optimization problem: to find the closest distribution to the exact posterior over some family of distributions. For practical reasons, the family of distributions in VI is usually constrained so that it does not include the exact posterior, even as a limit point. Thus, no matter how long VI is run, the resulting approximation will not approach the exact posterior. We propose to instead consider a more flexible approximating family consisting of all possible finite mixtures of a parametric base distribution (e.g., Gaussian). For efficient inference, we borrow ideas from gradient boosting to develop an algorithm we call boosting variational inference (BVI). BVI iteratively improves the current approximation by mixing it with a new component from the base distribution family and thereby yields progressively more accurate posterior approximations as more computing time is spent. Unlike a number of common VI variants including mean-field VI, BVI is able to capture multimodality, general posterior covariance, and nonstandard posterior shapes.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/20/2016

Variational Boosting: Iteratively Refining Posterior Approximations

We propose a black-box variational inference method to approximate intra...
research
06/04/2019

Universal Boosting Variational Inference

Boosting variational inference (BVI) approximates an intractable probabi...
research
05/19/2021

Boosting Variational Inference With Locally Adaptive Step-Sizes

Variational Inference makes a trade-off between the capacity of the vari...
research
07/15/2023

Variational Inference with Gaussian Score Matching

Variational inference (VI) is a method to approximate the computationall...
research
10/18/2021

Interpolating between sampling and variational inference with infinite stochastic mixtures

Sampling and Variational Inference (VI) are two large families of method...
research
04/13/2016

Variational Bayesian Inference of Line Spectra

In this paper, we address the fundamental problem of line spectral estim...
research
04/15/2019

Copula-like Variational Inference

This paper considers a new family of variational distributions motivated...

Please sign up or login with your details

Forgot password? Click here to reset