Deep neural networks using a single neuron: folded-in-time architecture using feedback-modulated delay loops.

Area of Science:

• Machine Learning
• Computational Neuroscience
• Artificial Intelligence

Background:

• Deep neural networks (DNNs) are powerful machine learning tools with widespread applications.
• Implementing large DNNs can be computationally intensive and complex.

Purpose of the Study:

• To present a method for folding deep neural networks into a single neuron with time-delayed feedback loops.
• To demonstrate that this single-neuron architecture can effectively represent and implement DNNs.

Main Methods:

• Developed a Folded-in-time DNN (Fit-DNN) architecture using a single neuron with multiple time-delayed feedback loops.
• Adjusted feedback modulations to adapt network connection weights.
• Utilized a back-propagation algorithm considering both delay-induced and local connections.

Main Results:

• The Fit-DNN architecture can fully represent standard and sparse DNNs.
• Network states emerge dynamically through the temporal unfolding of the neuron's dynamics.
• The Fit-DNN method demonstrated promising performance on benchmark tasks.

Conclusions:

• The Fit-DNN approach offers a novel dynamical systems implementation of DNNs.
• This method simplifies DNNs into a single-neuron model with feedback.
• The Fit-DNN shows potential for efficient and effective implementation of deep learning models.