Gradient-based Adaptive Markov Chain Monte Carlo

by   Michalis K. Titsias, et al.

We introduce a gradient-based learning method to automatically adapt Markov chain Monte Carlo (MCMC) proposal distributions to intractable targets. We define a maximum entropy regularised objective function, referred to as generalised speed measure, which can be robustly optimised over the parameters of the proposal distribution by applying stochastic gradient optimisation. An advantage of our method compared to traditional adaptive MCMC methods is that the adaptation occurs even when candidate state values are rejected. This is a highly desirable property of any adaptation strategy because the adaptation starts in early iterations even if the initial proposal distribution is far from optimum. We apply the framework for learning multivariate random walk Metropolis and Metropolis-adjusted Langevin proposals with full covariance matrices, and provide empirical evidence that our method can outperform other MCMC algorithms, including Hamiltonian Monte Carlo schemes.


page 7

page 15

page 16

page 17


Entropy-based adaptive Hamiltonian Monte Carlo

Hamiltonian Monte Carlo (HMC) is a popular Markov Chain Monte Carlo (MCM...

On the robustness of gradient-based MCMC algorithms

We analyse the tension between robustness and efficiency for Markov chai...

Learning variational autoencoders via MCMC speed measures

Variational autoencoders (VAEs) are popular likelihood-based generative ...

Geometrically adapted Langevin dynamics for Markov chain Monte Carlo simulations

Markov Chain Monte Carlo (MCMC) is one of the most powerful methods to s...

Automatic adaptation of MCMC algorithms

Markov chain Monte Carlo (MCMC) methods are ubiquitous tools for simulat...

Infinite dimensional adaptive MCMC for Gaussian processes

Latent Gaussian processes are widely applied in many fields like, statis...

Focusing on Difficult Directions for Learning HMC Trajectory Lengths

Hamiltonian Monte Carlo (HMC) is a premier Markov Chain Monte Carlo (MCM...

Please sign up or login with your details

Forgot password? Click here to reset