Scalable Gaussian Process Inference with Finite-data Mean and Variance Guarantees

by   Jonathan H. Huggins, et al.

Gaussian processes (GPs) offer a flexible class of priors for nonparametric Bayesian regression, but popular GP posterior inference methods are typically prohibitively slow or lack desirable finite-data guarantees on quality. We develop an approach to scalable approximate GP regression with finite-data guarantees on the accuracy of pointwise posterior mean and variance estimates. Our main contribution is a novel objective for approximate inference in the nonparametric setting: the "preconditioned Fisher (pF) divergence." We show that unlike popular divergences such as Kullback--Leibler (used in variational inference), the pF divergence bounds the 2-Wasserstein distance, which in turn bounds the pointwise difference of the mean and variance functions. We demonstrate that, for sparse GP likelihood approximations, we can minimize the pF divergence efficiently. Our experiments show that optimizing the pF divergence has the same computational requirements as variational sparse GPs while providing comparable empirical performance---in addition to our novel finite-data quality guarantees.


page 1

page 2

page 3

page 4


Structured Variational Inference for Coupled Gaussian Processes

Sparse variational approximations allow for principled and scalable infe...

Wasserstein-Splitting Gaussian Process Regression for Heterogeneous Online Bayesian Inference

Gaussian processes (GPs) are a well-known nonparametric Bayesian inferen...

Radial Neighbors for Provably Accurate Scalable Approximations of Gaussian Processes

In geostatistical problems with massive sample size, Gaussian processes ...

Sparse Gaussian Process Regression Beyond Variational Inference

The combination of inducing point methods with stochastic variational in...

Knot Selection in Sparse Gaussian Processes with a Variational Objective

Sparse, knot-based Gaussian processes have enjoyed considerable success ...

Quantile Propagation for Wasserstein-Approximate Gaussian Processes

In this work, we develop a new approximation method to solve the analyti...

Learning Gaussian Processes by Minimizing PAC-Bayesian Generalization Bounds

Gaussian Processes (GPs) are a generic modelling tool for supervised lea...

Please sign up or login with your details

Forgot password? Click here to reset