DeepAI AI Chat
Log In Sign Up

Copula-like Variational Inference

by   Marcel Hirt, et al.

This paper considers a new family of variational distributions motivated by Sklar's theorem. This family is based on new copula-like densities on the hypercube with non-uniform marginals which can be sampled efficiently, i.e. with a complexity linear in the dimension of state space. Then, the proposed variational densities that we suggest can be seen as arising from these copula-like densities used as base distributions on the hypercube with Gaussian quantile functions and sparse rotation matrices as normalizing flows. The latter correspond to a rotation of the marginals with complexity O(d d). We provide some empirical evidence that such a variational family can also approximate non-Gaussian posteriors and can be beneficial compared to Gaussian approximations. Our method performs largely comparably to state-of-the-art variational approximations on standard regression and classification benchmarks for Bayesian Neural Networks.


page 1

page 2

page 3

page 4


Structured variational approximations with skew normal decomposable graphical models

Although there is much recent work developing flexible variational metho...

Constructing a variational family for nonlinear state-space models

We consider the problem of maximum likelihood parameter estimation for n...

Boosting Variational Inference

Variational inference (VI) provides fast approximations of a Bayesian po...

Fast and Simple Natural-Gradient Variational Inference with Mixture of Exponential-family Approximations

Natural-gradient methods enable fast and simple algorithms for variation...

Variational Inference with Numerical Derivatives: variance reduction through coupling

The Black Box Variational Inference (Ranganath et al. (2014)) algorithm ...

Amortized backward variational inference in nonlinear state-space models

We consider the problem of state estimation in general state-space model...

Skewed Bernstein-von Mises theorem and skew-modal approximations

Deterministic Gaussian approximations of intractable posterior distribut...