Reservoir computing and extreme learning machines for non-linear time-series data analysis.

Area of Science:

• Computational neuroscience
• Machine learning

Background:

• Traditional neural network training is complex and computationally intensive, especially for recurrent networks.
• Random projection architectures like Echo State Networks (ESNs) and Extreme Learning Machines (ELMs) simplify training by only adjusting output weights.
• ESNs face a trade-off between non-linear mapping and short-term memory for highly non-linear time-series data.

Purpose of the Study:

• To investigate a new architecture, Reservoir with Random Static Projections (R(2)SP), to overcome the ESN trade-off.
• To evaluate a time-delay Extreme Learning Machine (TD-ELM) for enhanced performance on time-series tasks.
• To compare the performance of ESN, R(2)SP, and TD-ELM on a novel task with controllable non-linearity and short-term memory.

Main Methods:

• Implemented and evaluated the Reservoir with Random Static Projections (R(2)SP) architecture.
• Developed and tested a time-delay Extreme Learning Machine (TD-ELM).
• Utilized a novel benchmark task designed to vary short-term memory and non-linearity.

Main Results:

• The R(2)SP architecture showed significant performance improvements over standard ESNs.
• The TD-ELM demonstrated superior performance, outperforming both ESN and R(2)SP.
• The TD-ELM's hard-limiting memory was particularly effective for the tested data.

Conclusions:

• The TD-ELM architecture offers a significant advancement for processing non-linear time-series data with specific memory requirements.
• While TD-ELM excels in certain scenarios, ESN-based methods might be preferable for tasks demanding longer fading memory.