Learning ODEs via Diffeomorphisms for Fast and Robust Integration

by   Weiming Zhi, et al.

Advances in differentiable numerical integrators have enabled the use of gradient descent techniques to learn ordinary differential equations (ODEs). In the context of machine learning, differentiable solvers are central for Neural ODEs (NODEs), a class of deep learning models with continuous depth, rather than discrete layers. However, these integrators can be unsatisfactorily slow and inaccurate when learning systems of ODEs from long sequences, or when solutions of the system vary at widely different timescales in each dimension. In this paper we propose an alternative approach to learning ODEs from data: we represent the underlying ODE as a vector field that is related to another base vector field by a differentiable bijection, modelled by an invertible neural network. By restricting the base ODE to be amenable to integration, we can drastically speed up and improve the robustness of integration. We demonstrate the efficacy of our method in training and evaluating continuous neural networks models, as well as in learning benchmark ODE systems. We observe improvements of up to two orders of magnitude when integrating learned ODEs with GPUs computation.


On Numerical Integration in Neural Ordinary Differential Equations

The combination of ordinary differential equations and neural networks, ...

DiffEqFlux.jl - A Julia Library for Neural Differential Equations

DiffEqFlux.jl is a library for fusing neural networks and differential e...

TorchDyn: A Neural Differential Equations Library

Continuous-depth learning has recently emerged as a novel perspective on...

Delay Differential Neural Networks

Neural ordinary differential equations (NODEs) treat computation of inte...

Training Generative Adversarial Networks by Solving Ordinary Differential Equations

The instability of Generative Adversarial Network (GAN) training has fre...

Learning discrete Lagrangians for variationalPDEs from data and detection of travelling waves

The article shows how to learn models of dynamical systems from data whi...

Learning Dynamics from Noisy Measurements using Deep Learning with a Runge-Kutta Constraint

Measurement noise is an integral part while collecting data of a physica...

Please sign up or login with your details

Forgot password? Click here to reset