Variational Inference for Uncertainty on the Inputs of Gaussian Process Models

by   Andreas C. Damianou, et al.
The University of Sheffield
Athens University of Economics and Business

The Gaussian process latent variable model (GP-LVM) provides a flexible approach for non-linear dimensionality reduction that has been widely applied. However, the current approach for training GP-LVMs is based on maximum likelihood, where the latent projection variables are maximized over rather than integrated out. In this paper we present a Bayesian method for training GP-LVMs by introducing a non-standard variational inference framework that allows to approximately integrate out the latent variables and subsequently train a GP-LVM by maximizing an analytic lower bound on the exact marginal likelihood. We apply this method for learning a GP-LVM from iid observations and for learning non-linear dynamical systems where the observations are temporally correlated. We show that a benefit of the variational Bayesian procedure is its robustness to overfitting and its ability to automatically select the dimensionality of the nonlinear latent space. The resulting framework is generic, flexible and easy to extend for other purposes, such as Gaussian process regression with uncertain inputs and semi-supervised Gaussian processes. We demonstrate our method on synthetic data and standard machine learning benchmarks, as well as challenging real world datasets, including high resolution video data.


page 22

page 26

page 27

page 28

page 30


Generalised Gaussian Process Latent Variable Models (GPLVM) with Stochastic Variational Inference

Gaussian process latent variable models (GPLVM) are a flexible and non-l...

Spike and Slab Gaussian Process Latent Variable Models

The Gaussian process latent variable model (GP-LVM) is a popular approac...

Mixed Likelihood Gaussian Process Latent Variable Model

We present the Mixed Likelihood Gaussian process latent variable model (...

Mapping Interstellar Dust with Gaussian Processes

Interstellar dust corrupts nearly every stellar observation, and account...

Inverse Reinforcement Learning via Deep Gaussian Process

We propose a new approach to inverse reinforcement learning (IRL) based ...

Semi-described and semi-supervised learning with Gaussian processes

Propagating input uncertainty through non-linear Gaussian process (GP) m...

Please sign up or login with your details

Forgot password? Click here to reset