Improving Explorability in Variational Inference with Annealed Variational Objectives

by   Chin-Wei Huang, et al.
Element AI Inc
Université de Montréal

Despite the advances in the representational capacity of approximate distributions for variational inference, the optimization process can still limit the density that is ultimately learned. We demonstrate the drawbacks of biasing the true posterior to be unimodal, and introduce Annealed Variational Objectives (AVO) into the training of hierarchical variational methods. Inspired by Annealed Importance Sampling, the proposed method facilitates learning by incorporating energy tempering into the optimization objective. In our experiments, we demonstrate our method's robustness to deterministic warm up, and the benefits of encouraging exploration in the latent space.


Improving Importance Weighted Auto-Encoders with Annealed Importance Sampling

Stochastic variational inference with an amortized inference model and t...

Operator Variational Inference

Variational inference is an umbrella term for algorithms which cast Baye...

Moment-Based Variational Inference for Markov Jump Processes

We propose moment-based variational inference as a flexible framework fo...

Latent SDEs on Homogeneous Spaces

We consider the problem of variational Bayesian inference in a latent va...

Automatic Differentiation Variational Inference with Mixtures

Automatic Differentiation Variational Inference (ADVI) is a useful tool ...

Importance Weighting and Varational Inference

Recent work used importance sampling ideas for better variational bounds...

GFlowNets and variational inference

This paper builds bridges between two families of probabilistic algorith...

Please sign up or login with your details

Forgot password? Click here to reset