Randomness and isometries in echo state networks and compressed sensing
share
Although largely different concepts, echo state networks and compressed sensing models both rely on collections of random weights; as the reservoir dynamics for echo state networks, and the sensing coefficients in compressed sensing. Several methods for generating the random matrices and metrics to indicate desirable performance are well-studied in compressed sensing, but less so for echo state networks. This work explores any overlap in these compressed sensing methods and metrics for application to echo state networks. Several methods for generating the random reservoir weights are considered, and a new metric, inspired by the restricted isometry property for compressed sensing, is proposed for echo state networks. The methods and metrics are investigated theoretically and experimentally, with results suggesting that the same types of random matrices work well for both echo state network and compressed sensing scenarios, and that echo state network classification accuracy is improved when the proposed restricted isometry-like constants are close to 1.
READ FULL TEXT