Hierarchical Importance Weighted Autoencoders

by   Chin-Wei Huang, et al.

Importance weighted variational inference (Burda et al., 2015) uses multiple i.i.d. samples to have a tighter variational lower bound. We believe a joint proposal has the potential of reducing the number of redundant samples, and introduce a hierarchical structure to induce correlation. The hope is that the proposals would coordinate to make up for the error made by one another to reduce the variance of the importance estimator. Theoretically, we analyze the condition under which convergence of the estimator variance can be connected to convergence of the lower bound. Empirically, we confirm that maximization of the lower bound does implicitly minimize variance. Further analysis shows that this is a result of negative correlation induced by the proposed hierarchical meta sampling scheme, and performance of inference also improves when the number of samples increases.


page 3

page 6

page 11

page 15


Reinterpreting Importance-Weighted Autoencoders

The standard interpretation of importance-weighted autoencoders is that ...

Doubly Reparameterized Gradient Estimators for Monte Carlo Objectives

Deep latent variable models have become a popular model choice due to th...

Doubly Reparameterized Importance Weighted Structure Learning for Scene Graph Generation

As a structured prediction task, scene graph generation, given an input ...

U-Statistics for Importance-Weighted Variational Inference

We propose the use of U-statistics to reduce variance for gradient estim...

On improving Neymanian analysis for 2^K factorial designs with binary outcomes

2^K factorial designs are widely adopted by statisticians and the broade...

Importance Weighted Structure Learning for Scene Graph Generation

Scene graph generation is a structured prediction task aiming to explici...

The Evidence Lower Bound of Variational Autoencoders Converges to a Sum of Three Entropies

The central objective function of a variational autoencoder (VAE) is its...

Please sign up or login with your details

Forgot password? Click here to reset