Sparse Orthogonal Variational Inference for Gaussian Processes

by   Jiaxin Shi, et al.

We introduce a new interpretation of sparse variational approximations for Gaussian processes using inducing points which can lead to more scalable algorithms than previous methods. It is based on decomposing a Gaussian process as a sum of two independent processes: one in the subspace spanned by the inducing basis and the other in the orthogonal complement to this subspace. We show that this formulation recovers existing approximations and at the same time allows to obtain tighter lower bounds on the marginal likelihood and new stochastic variational inference algorithms. We demonstrate the efficiency of these algorithms in several Gaussian process models ranging from standard regression to multi-class classification using (deep) convolutional Gaussian processes and report state-of-the-art results on CIFAR-10 with purely GP-based models.


page 1

page 2

page 3

page 4


Gaussian Processes for Big Data

We introduce stochastic variational inference for Gaussian process model...

Quadruply Stochastic Gaussian Processes

We introduce a stochastic variational inference procedure for training s...

Neural Likelihoods for Multi-Output Gaussian Processes

We construct flexible likelihoods for multi-output Gaussian process mode...

Efficient Transformed Gaussian Processes for Non-Stationary Dependent Multi-class Classification

This work introduces the Efficient Transformed Gaussian Process (ETGP), ...

Scalable Variational Gaussian Processes via Harmonic Kernel Decomposition

We introduce a new scalable variational Gaussian process approximation w...

Fully Scalable Gaussian Processes using Subspace Inducing Inputs

We introduce fully scalable Gaussian processes, an implementation scheme...

On Sparse variational methods and the Kullback-Leibler divergence between stochastic processes

The variational framework for learning inducing variables (Titsias, 2009...

Code Repositories


Sparse Orthogonal Variational Inference for Gaussian Processes (SOLVE-GP)

view repo

Please sign up or login with your details

Forgot password? Click here to reset