DeepAI AI Chat
Log In Sign Up

DG-LMC: A Turn-key and Scalable Synchronous Distributed MCMC Algorithm via Langevin Monte Carlo within Gibbs

by   Vincent Plassier, et al.

Performing reliable Bayesian inference on a big data scale is becoming a keystone in the modern era of machine learning. A workhorse class of methods to achieve this task are Markov chain Monte Carlo (MCMC) algorithms and their design to handle distributed datasets has been the subject of many works. However, existing methods are not completely either reliable or computationally efficient. In this paper, we propose to fill this gap in the case where the dataset is partitioned and stored on computing nodes within a cluster under a master/slaves architecture. We derive a user-friendly centralised distributed MCMC algorithm with provable scaling in high-dimensional settings. We illustrate the relevance of the proposed methodology on both synthetic and real data experiments.


Challenges in Bayesian inference via Markov chain Monte Carlo for neural networks

Markov chain Monte Carlo (MCMC) methods and neural networks are instrume...

Variational Hamiltonian Monte Carlo via Score Matching

Traditionally, the field of computational Bayesian statistics has been d...

Computing Bayes: From Then 'Til Now'

This paper takes the reader on a journey through the history of Bayesian...

Monte Carlo Fusion

This paper proposes a new theory and methodology to tackle the problem o...

Bayesian Fusion: Scalable unification of distributed statistical analyses

There has recently been considerable interest in addressing the problem ...

Sequential Gibbs Sampling Algorithm for Cognitive Diagnosis Models with Many Attributes

Cognitive diagnosis models (CDMs) are useful statistical tools to provid...

Convergence of Dirichlet Forms for MCMC Optimal Scaling with General Target Distributions on Large Graphs

Markov chain Monte Carlo (MCMC) algorithms have played a significant rol...