Hierarchy-guided Model Selection for Time Series Forecasting

by   Arindam Jati, et al.

Generalizability of time series forecasting models depends on the quality of model selection. Temporal cross validation (TCV) is a standard technique to perform model selection in forecasting tasks. TCV sequentially partitions the training time series into train and validation windows, and performs hyperparameter optmization (HPO) of the forecast model to select the model with the best validation performance. Model selection with TCV often leads to poor test performance when the test data distribution differs from that of the validation data. We propose a novel model selection method, H-Pro that exploits the data hierarchy often associated with a time series dataset. Generally, the aggregated data at the higher levels of the hierarchy show better predictability and more consistency compared to the bottom-level data which is more sparse and (sometimes) intermittent. H-Pro performs the HPO of the lowest-level student model based on the test proxy forecasts obtained from a set of teacher models at higher levels in the hierarchy. The consistency of the teachers' proxy forecasts help select better student models at the lowest-level. We perform extensive empirical studies on multiple datasets to validate the efficacy of the proposed method. H-Pro along with off-the-shelf forecasting models outperform existing state-of-the-art forecasting methods including the winning models of the M5 point-forecasting competition.


page 1

page 2

page 3

page 4


Model selection in reconciling hierarchical time series

Model selection has been proven an effective strategy for improving accu...

Model Selection for Time Series Forecasting: Empirical Analysis of Different Estimators

Evaluating predictive models is a crucial task in predictive analytics. ...

Adaptive Learning on Time Series: Method and Financial Applications

We formally introduce a time series statistical learning method, called ...

Multi-Objective Model Selection for Time Series Forecasting

Research on time series forecasting has predominantly focused on develop...

Empirical Quantitative Analysis of COVID-19 Forecasting Models

COVID-19 has been a public health emergency of international concern sin...

Model combinations through revised base-rates

Standard selection criteria for forecasting models focus on information ...

A Worrying Analysis of Probabilistic Time-series Models for Sales Forecasting

Probabilistic time-series models become popular in the forecasting field...

Please sign up or login with your details

Forgot password? Click here to reset