On Robustness of Neural Ordinary Differential Equations

by   Hanshu Yan, et al.

Neural ordinary differential equations (ODEs) have been attracting increasing attention in various research domains recently. There have been some works studying optimization issues and approximation capabilities of neural ODEs, but their robustness is still yet unclear. In this work, we fill this important gap by exploring robustness properties of neural ODEs both empirically and theoretically. We first present an empirical study on the robustness of the neural ODE-based networks (ODENets) by exposing them to inputs with various types of perturbations and subsequently investigating the changes of the corresponding outputs. In contrast to conventional convolutional neural networks (CNNs), we find that the ODENets are more robust against both random Gaussian perturbations and adversarial attack examples. We then provide an insightful understanding of this phenomenon by exploiting a certain desirable property of the flow of a continuous-time ODE, namely that integral curves are non-intersecting. Our work suggests that, due to their intrinsic robustness, it is promising to use neural ODEs as a basic block for building robust deep network models. To further enhance the robustness of vanilla neural ODEs, we propose the time-invariant steady neural ODE (TisODE), which regularizes the flow on perturbed data via the time-invariant property and the imposition of a steady-state constraint. We show that the TisODE method outperforms vanilla neural ODEs and also can work in conjunction with other state-of-the-art architectural methods to build more robust deep networks.


page 1

page 2

page 3

page 4


Unconstrained Parametrization of Dissipative and Contracting Neural Ordinary Differential Equations

In this work, we introduce and study a class of Deep Neural Networks (DN...

Interpolation between Residual and Non-Residual Networks

Although ordinary differential equations (ODEs) provide insights for des...

ACE-NODE: Attentive Co-Evolving Neural Ordinary Differential Equations

Neural ordinary differential equations (NODEs) presented a new paradigm ...

Are Adversarial Robustness and Common Perturbation Robustness Independant Attributes ?

Neural Networks have been shown to be sensitive to common perturbations ...

Towards Sustainable Architecture: 3D Convolutional Neural Networks for Computational Fluid Dynamics Simulation and Reverse DesignWorkflow

We present a general and flexible approximation model for near real-time...

Robustness on Networks

We adopt the statistical framework on robustness proposed by Watson and ...

Please sign up or login with your details

Forgot password? Click here to reset