Learning Without Feedback: Fixed Random Learning Signals Allow for Feedforward Training of Deep Neural Networks.

Area of Science:

• Computational Neuroscience
• Machine Learning
• Edge Computing

Background:

• Traditional deep neural network training using backpropagation faces limitations due to bidirectional weight transport and update locking.
• These constraints hinder biological plausibility and pose challenges for developing efficient, low-cost adaptive smart sensors at the edge.
• Existing methods require significant memory access and buffering, increasing computational and memory overhead.

Purpose of the Study:

• To propose a biologically plausible and computationally efficient alternative to backpropagation for training deep neural networks.
• To address the limitations of weight transport and update locking in neural network training.
• To develop a method suitable for adaptive edge computing devices with relaxed computational and memory requirements.

Main Methods:

• The study views one-hot-encoded labels (targets) as a proxy for the error sign in supervised classification.
• Fixed random projections of these targets are used to enable layerwise feedforward training of hidden layers.
• The proposed Direct Random Target Projection (DRTP) algorithm is introduced and evaluated.

Main Results:

• DRTP successfully solves the weight transport and update locking problems inherent in backpropagation.
• The algorithm significantly relaxes computational and memory requirements, making it suitable for resource-constrained environments.
• DRTP demonstrates a favorable trade-off between classification accuracy and computational cost for edge devices.

Conclusions:

• Direct Random Target Projection (DRTP) offers a viable and efficient alternative to backpropagation for training deep neural networks.
• The DRTP algorithm is particularly well-suited for adaptive edge computing applications due to its reduced computational and memory footprint.
• This approach advances the development of biologically plausible and practical AI systems for edge devices.