Learning Deep Mixtures of Gaussian Process Experts Using Sum-Product Networks

by   Martin Trapp, et al.

While Gaussian processes (GPs) are the method of choice for regression tasks, they also come with practical difficulties, as inference cost scales cubic in time and quadratic in memory. In this paper, we introduce a natural and expressive way to tackle these problems, by incorporating GPs in sum-product networks (SPNs), a recently proposed tractable probabilistic model allowing exact and efficient inference. In particular, by using GPs as leaves of an SPN we obtain a novel flexible prior over functions, which implicitly represents an exponentially large mixture of local GPs. Exact and efficient posterior inference in this model can be done in a natural interplay of the inference mechanisms in GPs and SPNs. Thereby, each GP is -- similarly as in a mixture of experts approach -- responsible only for a subset of data points, which effectively reduces inference cost in a divide and conquer fashion. We show that integrating GPs into the SPN framework leads to a promising probabilistic regression model which is: (1) computational and memory efficient, (2) allows efficient and exact posterior inference, (3) is flexible enough to mix different kernel functions, and (4) naturally accounts for non-stationarities in time series. In a variate of experiments, we show that the SPN-GP model can learn input dependent parameters and hyper-parameters and is on par with or outperforms the traditional GPs as well as state of the art approximations on real-world data.


page 1

page 2

page 3

page 4


Deep Structured Mixtures of Gaussian Processes

Gaussian Processes (GPs) are powerful non-parametric Bayesian regression...

Overlapping Mixtures of Gaussian Processes for the Data Association Problem

In this work we introduce a mixture of GPs to address the data associati...

When Gaussian Process Meets Big Data: A Review of Scalable GPs

The vast quantity of information brought by big data as well as the evol...

Correlated Product of Experts for Sparse Gaussian Process Regression

Gaussian processes (GPs) are an important tool in machine learning and s...

Leveraging Probabilistic Circuits for Nonparametric Multi-Output Regression

Inspired by recent advances in the field of expert-based approximations ...

Scalable Gaussian Process Inference with Stan

Gaussian processes (GPs) are sophisticated distributions to model functi...

Neural Processes

A neural network (NN) is a parameterised function that can be tuned via ...

Please sign up or login with your details

Forgot password? Click here to reset