DeepAI AI Chat
Log In Sign Up

Unbiased Bayes for Big Data: Paths of Partial Posteriors

by   Heiko Strathmann, et al.
University of Warwick

A key quantity of interest in Bayesian inference are expectations of functions with respect to a posterior distribution. Markov Chain Monte Carlo is a fundamental tool to consistently compute these expectations via averaging samples drawn from an approximate posterior. However, its feasibility is being challenged in the era of so called Big Data as all data needs to be processed in every iteration. Realising that such simulation is an unnecessarily hard problem if the goal is estimation, we construct a computationally scalable methodology that allows unbiased estimation of the required expectations -- without explicit simulation from the full posterior. The scheme's variance is finite by construction and straightforward to control, leading to algorithms that are provably unbiased and naturally arrive at a desired error tolerance. This is achieved at an average computational complexity that is sub-linear in the size of the dataset and its free parameters are easy to tune. We demonstrate the utility and generality of the methodology on a range of common statistical models applied to large-scale benchmark and real-world datasets.


page 1

page 2

page 3

page 4


On Markov chain Monte Carlo methods for tall data

Markov chain Monte Carlo methods are often deemed too computationally in...

SwISS: A Scalable Markov chain Monte Carlo Divide-and-Conquer Strategy

Divide-and-conquer strategies for Monte Carlo algorithms are an increasi...

Invertible Flow Non Equilibrium sampling

Simultaneously sampling from a complex distribution with intractable nor...

Unbiased Parameter Inference for a Class of Partially Observed Levy-Process Models

We consider the problem of static Bayesian inference for partially obser...

Unbiased approximations of products of expectations

We consider the problem of approximating the product of n expectations w...

Measuring the accuracy of likelihood-free inference

Complex scientific models where the likelihood cannot be evaluated prese...