Learning in the machine: Recirculation is random backpropagation.

Area of Science:

• Computational Neuroscience
• Machine Learning
• Deep Learning Architectures

Background:

• Physical neural systems require local learning rules (space and time).
• Optimal deep learning necessitates non-local information at synapses.
• Standard deep learning requires a separate backward channel for non-local information.

Purpose of the Study:

• To identify and study local learning algorithms for circular autoencoders.
• To investigate if forward activation channels can serve as deep learning channels.
• To enable local learning without additional deep learning channels.

Main Methods:

• Systematic identification and classification of recirculation-based algorithms.
• Mathematical derivations to analyze algorithm convergence.
• Simulations to test robustness and performance.

Main Results:

• Several local learning algorithms based on recirculating output information were identified.
• These algorithms are robust and converge to critical points of the global error function.
• Recirculation algorithms approximate an adaptive form of random backpropagation.

Conclusions:

• Local learning in circular autoencoders is achievable without dedicated backward channels.
• Recirculating information via forward channels offers an efficient alternative for deep learning.
• The proposed algorithms provide a viable approach for biologically plausible learning in neural networks.