Latent Time Neural Ordinary Differential Equations

by   Srinivas Anumasa, et al.

Neural ordinary differential equations (NODE) have been proposed as a continuous depth generalization to popular deep learning models such as Residual networks (ResNets). They provide parameter efficiency and automate the model selection process in deep learning models to some extent. However, they lack the much-required uncertainty modelling and robustness capabilities which are crucial for their use in several real-world applications such as autonomous driving and healthcare. We propose a novel and unique approach to model uncertainty in NODE by considering a distribution over the end-time T of the ODE solver. The proposed approach, latent time NODE (LT-NODE), treats T as a latent variable and apply Bayesian learning to obtain a posterior distribution over T from the data. In particular, we use variational inference to learn an approximate posterior and the model parameters. Prediction is done by considering the NODE representations from different samples of the posterior and can be done efficiently using a single forward pass. As T implicitly defines the depth of a NODE, posterior distribution over T would also help in model selection in NODE. We also propose, adaptive latent time NODE (ALT-NODE), which allow each data point to have a distinct posterior distribution over end-times. ALT-NODE uses amortized variational inference to learn an approximate posterior using inference networks. We demonstrate the effectiveness of the proposed approaches in modelling uncertainty and robustness through experiments on synthetic and several real-world image classification data.


page 1

page 2

page 3

page 4


Improving Robustness and Uncertainty Modelling in Neural Ordinary Differential Equations

Neural ordinary differential equations (NODE) have been proposed as a co...

Infinitely Deep Bayesian Neural Networks with Stochastic Differential Equations

We perform scalable approximate inference in a recently-proposed family ...

Flat Seeking Bayesian Neural Networks

Bayesian Neural Networks (BNNs) offer a probabilistic interpretation for...

Model Selection for Ordinary Differential Equations: a Statistical Testing Approach

Ordinary differential equations (ODEs) are foundational in modeling intr...

Non-Volatile Memory Accelerated Posterior Estimation

Bayesian inference allows machine learning models to express uncertainty...

Constraining the Dynamics of Deep Probabilistic Models

We introduce a novel generative formulation of deep probabilistic models...

Loss function based second-order Jensen inequality and its application to particle variational inference

Bayesian model averaging, obtained as the expectation of a likelihood fu...

Please sign up or login with your details

Forgot password? Click here to reset