Transfer-RLS method and transfer-FORCE learning for simple and fast training of reservoir computing models

Area of Science:

• * Computational neuroscience and machine learning.
• * Development of energy-efficient hardware for artificial intelligence.

Background:

• * Reservoir computing, a type of recurrent neural network, shows promise for energy-efficient, real-time information processing hardware.
• * Existing online learning methods like Recursive Least Squares (RLS) and Least Mean Squares (LMS) present trade-offs between convergence speed and computational complexity.
• * Efficient online learning is crucial for low-power reservoir computing devices.

Purpose of the Study:

• * To develop a novel, efficient online learning method for reservoir computing models.
• * To bridge the performance gap between RLS and LMS methods.
• * To enable simpler operations and faster convergence for training reservoir computing hardware.

Main Methods:

• * Proposal of the transfer-RLS method, a hybrid approach between RLS and LMS.
• * Utilizing a pre-training phase to update the gain matrix, avoiding updates during main training.
• * Numerical and analytical validation of the transfer-RLS method's performance.
• * Development of a modified transfer-RLS method (transfer-FORCE learning) for closed-loop systems.

Main Results:

• * The transfer-RLS method demonstrates significantly faster convergence than the LMS method.
• * It achieves comparable convergence speed to RLS with substantially simpler operations.
• * The transfer-FORCE learning variant successfully applies to challenging closed-loop reservoir computing models.
• * The proposed method offers a practical solution for efficient training of physical reservoir computing hardware.

Conclusions:

• * The transfer-RLS method provides an effective and computationally efficient approach for online learning in reservoir computing.
• * This method enhances the feasibility of using low-power reservoir computing devices for real-time applications.
• * The transfer-FORCE learning extension broadens the applicability of reservoir computing to more complex dynamic systems.