Training Deep Surrogate Models with Large Scale Online Learning

by   Lucas Meyer, et al.

The spatiotemporal resolution of Partial Differential Equations (PDEs) plays important roles in the mathematical description of the world's physical phenomena. In general, scientists and engineers solve PDEs numerically by the use of computationally demanding solvers. Recently, deep learning algorithms have emerged as a viable alternative for obtaining fast solutions for PDEs. Models are usually trained on synthetic data generated by solvers, stored on disk and read back for training. This paper advocates that relying on a traditional static dataset to train these models does not allow the full benefit of the solver to be used as a data generator. It proposes an open source online training framework for deep surrogate models. The framework implements several levels of parallelism focused on simultaneously generating numerical simulations and training deep neural networks. This approach suppresses the I/O and storage bottleneck associated with disk-loaded datasets, and opens the way to training on significantly larger datasets. Experiments compare the offline and online training of four surrogate models, including state-of-the-art architectures. Results indicate that exposing deep surrogate models to more dataset diversity, up to hundreds of GB, can increase model generalization capabilities. Fully connected neural networks, Fourier Neural Operator (FNO), and Message Passing PDE Solver prediction accuracy is improved by 68


page 14

page 15

page 16

page 17


Distributed Machine Learning for Computational Engineering using MPI

We propose a framework for training neural networks that are coupled wit...

Fourier Neural Operator with Learned Deformations for PDEs on General Geometries

Deep learning surrogate models have shown promise in solving partial dif...

Finite Difference Nets: A Deep Recurrent Framework for Solving Evolution PDEs

There has been an arising trend of adopting deep learning methods to stu...

PyDEns: a Python Framework for Solving Differential Equations with Neural Networks

Recently, a lot of papers proposed to use neural networks to approximate...

A neural operator-based surrogate solver for free-form electromagnetic inverse design

Neural operators have emerged as a powerful tool for solving partial dif...

Physics-enhanced deep surrogates for PDEs

We present a "physics-enhanced deep-surrogate ("PEDS") approach towards ...

Learning the Solution Operator of Boundary Value Problems using Graph Neural Networks

As an alternative to classical numerical solvers for partial differentia...

Please sign up or login with your details

Forgot password? Click here to reset