Physical deep learning with biologically inspired training method: gradient-free approach for physical hardware.

Area of Science:

• Neuromorphic Engineering
• Artificial Intelligence
• Computational Neuroscience

Background:

• The increasing demand for artificial intelligence (AI) drives research into unconventional computation using physical devices.
• Current AI learning methods, like backpropagation, are optimized for digital systems and not suitable for physical, brain-inspired analog processors.
• A gap exists in training algorithms that can effectively utilize the capabilities of physical computing hardware for AI tasks.

Purpose of the Study:

• To present a novel physical deep learning method for training neural networks on unconventional hardware.
• To enable efficient training of physical neural networks without requiring knowledge of the system's physical properties or gradients.
• To demonstrate a practical approach for accelerating neuromorphic computation.

Main Methods:

• Extended the biologically inspired Direct Feedback Alignment (DFA) algorithm.
• Introduced random projection and alternative nonlinear activation functions for the DFA algorithm.
• Implemented and tested the method on an optoelectronic recurrent neural network (deep reservoir computer).

Main Results:

• Successfully trained a physical neural network without prior knowledge of its physical system or gradient.
• Demonstrated proof-of-concept using a deep reservoir computer, confirming accelerated computation potential.
• Achieved competitive performance on benchmark tasks, validating the proposed training methodology.

Conclusions:

• The developed physical deep learning approach offers a practical solution for training and accelerating neuromorphic computation.
• This method overcomes limitations of digital-centric training algorithms for physical computing systems.
• The findings pave the way for more efficient and scalable AI hardware.