DeepAI AI Chat
Log In Sign Up

Monte Carlo Variational Auto-Encoders

by   Achille Thin, et al.

Variational auto-encoders (VAE) are popular deep latent variable models which are trained by maximizing an Evidence Lower Bound (ELBO). To obtain tighter ELBO and hence better variational approximations, it has been proposed to use importance sampling to get a lower variance estimate of the evidence. However, importance sampling is known to perform poorly in high dimensions. While it has been suggested many times in the literature to use more sophisticated algorithms such as Annealed Importance Sampling (AIS) and its Sequential Importance Sampling (SIS) extensions, the potential benefits brought by these advanced techniques have never been realized for VAE: the AIS estimate cannot be easily differentiated, while SIS requires the specification of carefully chosen backward Markov kernels. In this paper, we address both issues and demonstrate the performance of the resulting Monte Carlo VAEs on a variety of applications.


Hamiltonian Variational Auto-Encoder

Variational Auto-Encoders (VAEs) have become very popular techniques to ...

Monte Carlo Inference via Greedy Importance Sampling

We present a new method for conducting Monte Carlo inference in graphica...

Improving Importance Weighted Auto-Encoders with Annealed Importance Sampling

Stochastic variational inference with an amortized inference model and t...

Multiple Importance Sampling ELBO and Deep Ensembles of Variational Approximations

In variational inference (VI), the marginal log-likelihood is estimated ...

Light Transport Simulation via Generalized Multiple Importance Sampling

Multiple importance sampling (MIS) is employed to reduce variance of est...

Tensor Monte Carlo: particle methods for the GPU era

Multi-sample objectives improve over single-sample estimates by giving t...

Exhaustive Neural Importance Sampling applied to Monte Carlo event generation

The generation of accurate neutrino-nucleus cross-section models needed ...