Deep Supervised Learning Using Local Errors.

Area of Science:

• Artificial Intelligence
• Computational Neuroscience
• Machine Learning

Background:

• Error backpropagation is effective for deep networks but biologically implausible due to delayed error signals.
• Biological neural networks require local learning rules, unlike the non-local dependencies in backpropagation.

Purpose of the Study:

• To propose a biologically plausible alternative to error backpropagation for training deep neural networks.
• To develop a local learning mechanism that reduces computational and memory overheads.

Main Methods:

• Proposed a new learning mechanism using fixed, random auxiliary classifiers for local error generation in each layer.
• Trained deep networks layer-by-layer or simultaneously using only local error information.
• Addressed biological plausibility concerns, including weight symmetry.

Main Results:

• The proposed local error mechanism achieved performance comparable to standard backpropagation on the CIFAR10 dataset.
• Outperformed biologically-motivated feedback alignment techniques.
• Demonstrated suitability for custom hardware, significantly reducing memory and communication overheads.

Conclusions:

• The local error mechanism offers a biologically plausible and efficient alternative to backpropagation for deep learning.
• This approach facilitates supervised learning of feature hierarchies and has practical implications for hardware acceleration.