Simultaneous perturbation learning rule for recurrent neural networks and its FPGA implementation

Area of Science:

• Artificial Intelligence
• Computational Neuroscience
• Machine Learning

Background:

• Recurrent neural networks (RNNs) excel at dynamic information processing but lack convenient learning schemes.
• Existing methods often struggle with analog or oscillatory learning paradigms.

Purpose of the Study:

• To introduce a novel recursive learning scheme for recurrent neural networks.
• To demonstrate its applicability to analog and oscillatory learning tasks.
• To explore hardware implementation of this scheme using Field-Programmable Gate Arrays (FPGAs).

Main Methods:

• Development of a recursive learning scheme for RNNs.
• Application of the simultaneous perturbation method for efficient gradient estimation.
• Hardware implementation of Hopfield neural networks on FPGAs.

Main Results:

• The proposed learning scheme effectively handles both analog and oscillatory learning for RNNs.
• Successful hardware implementation of Hopfield neural networks using FPGAs was achieved.
• Demonstrated feasibility of the learning scheme for dynamic information processing tasks.

Conclusions:

• The simultaneous perturbation-based recursive learning scheme offers a viable solution for RNN training.
• This approach broadens the applicability of RNNs to complex analog and oscillatory dynamics.
• FPGA implementation provides a practical pathway for real-world deployment of advanced RNN systems.