A Mini-Batch Method for Solving Nonlinear PDEs with Gaussian Processes

by   Xianjin Yang, et al.
California Institute of Technology

Gaussian processes (GPs) based methods for solving partial differential equations (PDEs) demonstrate great promise by bridging the gap between the theoretical rigor of traditional numerical algorithms and the flexible design of machine learning solvers. The main bottleneck of GP methods lies in the inversion of a covariance matrix, whose cost grows cubically concerning the size of samples. Drawing inspiration from neural networks, we propose a mini-batch algorithm combined with GPs to solve nonlinear PDEs. The algorithm takes a mini-batch of samples at each step to update the GP model. Thus, the computational cost is allotted to each iteration. Using stability analysis and convexity arguments, we show that the mini-batch method steadily reduces a natural measure of errors towards zero at the rate of O(1/K + 1/M), where K is the number of iterations and M is the batch size. Numerical results show that smooth problems benefit from a small batch size, while less regular problems require careful sample selection for optimal accuracy.


page 1

page 2

page 3

page 4


Sparse Gaussian processes for solving nonlinear PDEs

In this article, we propose a numerical method based on sparse Gaussian ...

Attentive Gaussian processes for probabilistic time-series generation

The transduction of sequence has been mostly done by recurrent networks,...

Numerical methods for Mean field Games based on Gaussian Processes and Fourier Features

In this article, we propose two numerical methods, the Gaussian Process ...

Batch-efficient EigenDecomposition for Small and Medium Matrices

EigenDecomposition (ED) is at the heart of many computer vision algorith...

Neural-net-induced Gaussian process regression for function approximation and PDE solution

Neural-net-induced Gaussian process (NNGP) regression inherits both the ...

Near-linear Time Gaussian Process Optimization with Adaptive Batching and Resparsification

Gaussian processes (GP) are one of the most successful frameworks to mod...

An analytic comparison of regularization methods for Gaussian Processes

Gaussian Processes (GPs) are a popular approach to predict the output of...

Please sign up or login with your details

Forgot password? Click here to reset