CrossPyramid: Neural Ordinary Differential Equations Architecture for Partially-observed Time-series

by   Futoon M. Abushaqra, et al.

Ordinary Differential Equations (ODE)-based models have become popular foundation models to solve many time-series problems. Combining neural ODEs with traditional RNN models has provided the best representation for irregular time series. However, ODE-based models require the trajectory of hidden states to be defined based on the initial observed value or the last available observation. This fact raises questions about how long the generated hidden state is sufficient and whether it is effective when long sequences are used instead of the typically used shorter sequences. In this article, we introduce CrossPyramid, a novel ODE-based model that aims to enhance the generalizability of sequences representation. CrossPyramid does not rely only on the hidden state from the last observed value; it also considers ODE latent representations learned from other samples. The main idea of our proposed model is to define the hidden state for the unobserved values based on the non-linear correlation between samples. Accordingly, CrossPyramid is built with three distinctive parts: (1) ODE Auto-Encoder to learn the best data representation. (2) Pyramidal attention method to categorize the learned representations (hidden state) based on the relationship characteristics between samples. (3) Cross-level ODE-RNN to integrate the previously learned information and provide the final latent state for each sample. Through extensive experiments on partially-observed synthetic and real-world datasets, we show that the proposed architecture can effectively model the long gaps in intermittent series and outperforms state-of-the-art approaches. The results show an average improvement of 10% on univariate and multivariate datasets for both forecasting and classification tasks.


Neural Controlled Differential Equations for Irregular Time Series

Neural ordinary differential equations are an attractive option for mode...

Deep Latent State Space Models for Time-Series Generation

Methods based on ordinary differential equations (ODEs) are widely used ...

Forecasting Irregularly Sampled Time Series using Graphs

Forecasting irregularly sampled time series with missing values is a cru...

Improved Batching Strategy For Irregular Time-Series ODE

Irregular time series data are prevalent in the real world and are chall...

Causally interpretable multi-step time series forecasting: A new machine learning approach using simulated differential equations

This work represents a new approach which generates then analyzes a high...

Latent Temporal Flows for Multivariate Analysis of Wearables Data

Increased use of sensor signals from wearable devices as rich sources of...

OCT-GAN: Neural ODE-based Conditional Tabular GANs

Synthesizing tabular data is attracting much attention these days for va...

Please sign up or login with your details

Forgot password? Click here to reset