Training Echo State Networks with Regularization through Dimensionality Reduction

by   Sigurd Løkse, et al.
University of Tromsø the Arctic University of Norway

In this paper we introduce a new framework to train an Echo State Network to predict real valued time-series. The method consists in projecting the output of the internal layer of the network on a space with lower dimensionality, before training the output layer to learn the target task. Notably, we enforce a regularization constraint that leads to better generalization capabilities. We evaluate the performances of our approach on several benchmark tests, using different techniques to train the readout of the network, achieving superior predictive performance when using the proposed framework. Finally, we provide an insight on the effectiveness of the implemented mechanics through a visualization of the trajectory in the phase space and relying on the methodologies of nonlinear time-series analysis. By applying our method on well known chaotic systems, we provide evidence that the lower dimensional embedding retains the dynamical properties of the underlying system better than the full-dimensional internal states of the network.


page 1

page 2

page 3

page 4


Model-Coupled Autoencoder for Time Series Visualisation

We present an approach for the visualisation of a set of time series tha...

Deep Temporal Clustering : Fully Unsupervised Learning of Time-Domain Features

Unsupervised learning of time series data, also known as temporal cluste...

Machine Learning of Time Series Using Time-delay Embedding and Precision Annealing

Tasking machine learning to predict segments of a time series requires e...

Deep learning of dynamical attractors from time series measurements

Experimental measurements of physical systems often have a finite number...

Efficient Optimization of Echo State Networks for Time Series Datasets

Echo State Networks (ESNs) are recurrent neural networks that only train...

Imaging the time series of one single referenced EEG electrode for Epileptic Seizures Risk Analysis

The time series captured by a single scalp electrode (plus the reference...

Please sign up or login with your details

Forgot password? Click here to reset