Effect of shapes of activation functions on predictability in the echo state network

05/22/2019

∙

We investigate prediction accuracy for time series of Echo state networks with respect to several kinds of activation functions. As a result, we found that some kinds of activation functions with an appropriate nonlinearity show high performance compared to the conventional sigmoid function.

READ FULL TEXT

Effect of shapes of activation functions on predictability in the echo state network

Orthogonal-Padé Activation Functions: Trainable Activation functions for smooth and faster convergence in deep networks

Normalized Activation Function: Toward Better Convergence

Evolving Parsimonious Networks by Mixing Activation Functions

Otimizacao de pesos e funcoes de ativacao de redes neurais aplicadas na previsao de series temporais

Squashing activation functions in benchmark tests: towards eXplainable Artificial Intelligence using continuous-valued logic

Lifted Bregman Training of Neural Networks

Central and Non-central Limit Theorems arising from the Scattering Transform and its Neural Activation Generalization

Effect of shapes of activation functions on predictability in the echo state network

Related Research

Orthogonal-Padé Activation Functions: Trainable Activation functions for smooth and faster convergence in deep networks

Normalized Activation Function: Toward Better Convergence

Evolving Parsimonious Networks by Mixing Activation Functions

Otimizacao de pesos e funcoes de ativacao de redes neurais aplicadas na previsao de series temporais

Squashing activation functions in benchmark tests: towards eXplainable Artificial Intelligence using continuous-valued logic

Lifted Bregman Training of Neural Networks

Central and Non-central Limit Theorems arising from the Scattering Transform and its Neural Activation Generalization