Related Experiment Videos
Efficient Training of Recurrent Neural Network with Time Delays
Emanuel Marom1, David Saad, Barak Cohen
1Tel Aviv University, UK
Summary
Training recurrent neural networks with time delays is challenging. A new adaptive simulated annealing (ASA) method offers a robust, fast, and tuning-free solution superior to other algorithms.
Area of Science:
- Artificial Intelligence
- Computational Neuroscience
- Machine Learning
Background:
- Training recurrent neural networks (RNNs) is complex.
- Synaptic delays in RNNs further complicate training.
- Improved network performance can outweigh training difficulties.
Purpose of the Study:
- To present a robust method for training RNNs with time delays.
- To introduce the adaptive simulated annealing (ASA) algorithm for this purpose.
- To demonstrate the efficacy of ASA on benchmark tests.
Main Methods:
- Utilizing the adaptive simulated annealing (ASA) algorithm.
- Implementing ASA for training RNNs with synaptic delays.
- Testing the algorithm on standard RNN benchmark datasets.
Main Results:
- ASA proved superior to other training algorithms.
- The method requires no parameter tuning.
- Training is efficient, suitable for personal computers.
Conclusions:
- ASA is an effective and robust algorithm for training RNNs with time delays.
- The algorithm's speed and ease of use make it practical.
- ASA offers a significant improvement over existing training methods.