Deep Learning With Asymmetric Connections and Hebbian Updates.

Area of Science:

• Deep Learning
• Computational Neuroscience
• Machine Learning

Background:

• Back-propagation (BP) is the standard for training deep neural networks.
• BP relies on symmetric feedforward and feedback weights, which is biologically implausible.
• Existing biologically inspired methods often show performance degradation with network depth.

Purpose of the Study:

• To investigate Hebbian learning rules for training deep neural networks.
• To develop a biologically plausible alternative to back-propagation.
• To assess the performance of Hebbian updates on challenging image datasets.

Main Methods:

• Implemented deep networks using locally updated Hebbian learning rules for both feedforward and feedback weights.
• Separated feedback and feedforward weights, updating them independently.
• Utilized untied weights in convolutional layers to enhance biological plausibility.
• Theoretically analyzed convergence in the linear case.

Main Results:

• Hebbian-trained networks achieved performance comparable to back-propagation on challenging image datasets.
• Separating and randomly initializing feedback weights, while updating them locally, maintained high performance.
• Untied convolutional layers yielded similar results to tied weights, improving biological plausibility.
• Theoretical analysis showed accelerated convergence with feedback weight updates in linear models.

Conclusions:

• Hebbian learning provides a viable and biologically plausible alternative to back-propagation for deep networks.
• Local learning rules and separated weights are key to achieving competitive performance.
• This approach offers a promising direction for more brain-like artificial intelligence.