Fast and Accurate Variational Inference for Models with Many Latent Variables

by   Ruben Loaiza-Maya, et al.

Models with a large number of latent variables are often used to fully utilize the information in big or complex data. However, they can be difficult to estimate using standard approaches, and variational inference methods are a popular alternative. Key to the success of these is the selection of an approximation to the target density that is accurate, tractable and fast to calibrate using optimization methods. Mean field or structured Gaussian approximations are common, but these can be inaccurate and slow to calibrate when there are many latent variables. Instead, we propose a family of tractable variational approximations that are more accurate and faster to calibrate for this case. The approximation is a parsimonious copula model for the parameter posterior, combined with the exact conditional posterior of the latent variables. We derive a simplified expression for the re-parameterization gradient of the variational lower bound, which is the main ingredient of efficient optimization algorithms used to implement variational estimation. We illustrate using two substantive econometric examples. The first is a nonlinear state space model for U.S. inflation. The second is a random coefficients tobit model applied to a rich marketing dataset with one million sales observations from a panel of 10,000 individuals. In both cases, we show that our approximating family is faster to calibrate than either mean field or structured Gaussian approximations, and that the gains in posterior estimation accuracy are considerable.


Copula variational inference

We develop a general variational inference method that preserves depende...

Conditionally structured variational Gaussian approximation with importance weights

We develop flexible methods of deriving variational inference for models...

Mean-Field Variational Inference for Gradient Matching with Gaussian Processes

Gradient matching with Gaussian processes is a promising tool for learni...

Amortized Variational Inference for Simple Hierarchical Models

It is difficult to use subsampling with variational inference in hierarc...

Natural Gradient Hybrid Variational Inference with Application to Deep Mixed Models

Stochastic models with global parameters θ and latent variables z are co...

A Structured Variational Auto-encoder for Learning Deep Hierarchies of Sparse Features

In this note we present a generative model of natural images consisting ...

Implicit copula variational inference

Key to effective generic, or "black-box", variational inference is the s...

Please sign up or login with your details

Forgot password? Click here to reset