Incorporating NODE with Pre-trained Neural Differential Operator for Learning Dynamics

by   Shiqi Gong, et al.

Learning dynamics governed by differential equations is crucial for predicting and controlling the systems in science and engineering. Neural Ordinary Differential Equation (NODE), a deep learning model integrated with differential equations, learns the dynamics directly from the samples on the trajectory and shows great promise in the scientific field. However, the training of NODE highly depends on the numerical solver, which can amplify numerical noise and be unstable, especially for ill-conditioned dynamical systems. In this paper, to reduce the reliance on the numerical solver, we propose to enhance the supervised signal in learning dynamics. Specifically, beyond learning directly from the trajectory samples, we pre-train a neural differential operator (NDO) to output an estimation of the derivatives to serve as an additional supervised signal. The NDO is pre-trained on a class of symbolic functions, and it learns the mapping between the trajectory samples of these functions to their derivatives. We provide theoretical guarantee on that the output of NDO can well approximate the ground truth derivatives by proper tuning the complexity of the library. To leverage both the trajectory signal and the estimated derivatives from NDO, we propose an algorithm called NDO-NODE, in which the loss function contains two terms: the fitness on the true trajectory samples and the fitness on the estimated derivatives that are output by the pre-trained NDO. Experiments on various of dynamics show that our proposed NDO-NODE can consistently improve the forecasting accuracy.


page 1

page 2

page 3

page 4


Optimizing differential equations to fit data and predict outcomes

Many scientific problems focus on observed patterns of change or on how ...

Stabilized Neural Differential Equations for Learning Constrained Dynamics

Many successful methods to learn dynamical systems from data have recent...

Learning Differential Equations that are Easy to Solve

Differential equations parameterized by neural networks become expensive...

Augmenting Neural Differential Equations to Model Unknown Dynamical Systems with Incomplete State Information

Neural Ordinary Differential Equations replace the right-hand side of a ...

LEARNEST: LEARNing Enhanced Model-based State ESTimation for Robots using Knowledge-based Neural Ordinary Differential Equations

State estimation is an important aspect in many robotics applications. I...

Neural Laplace: Learning diverse classes of differential equations in the Laplace domain

Neural Ordinary Differential Equations model dynamical systems with ODEs...

Confederated Modular Differential Equation APIs for Accelerated Algorithm Development and Benchmarking

Performant numerical solving of differential equations is required for l...

Please sign up or login with your details

Forgot password? Click here to reset