Contrastive Hebbian learning with random feedback weights.

Area of Science:

• Computational neuroscience
• Machine learning algorithms
• Artificial neural networks

Background:

• Contrastive Hebbian learning (CHL) is a powerful neural network training rule inspired by gradient backpropagation.
• CHL relies on synaptic weight symmetries, which lack biological evidence in the brain.
• Existing methods require specific symmetries for effective feedback signal transformation.

Purpose of the Study:

• To introduce a novel variant of contrastive Hebbian learning, termed random contrastive Hebbian learning (RCHL).
• To develop a biologically plausible learning algorithm that does not depend on synaptic weight symmetries.
• To investigate the impact of random matrices on neural network learning dynamics and performance.

Main Methods:

• Proposed RCHL algorithm utilizing random matrices for feedback signal transformation during the clamped phase.
• Described neural dynamics using first-order non-linear differential equations.
• Employed pseudospectra analysis to examine the influence of random matrices on learning.

Main Results:

• Experimentally verified RCHL on Boolean logic, handwritten digit/letter classification, and autoencoding tasks.
• Demonstrated that random matrices significantly impact learning parameters and process.
• Pseudospectra analysis provided insights into how random matrices affect learning dynamics.

Conclusions:

• RCHL provides a viable alternative to traditional CHL, removing the need for synaptic symmetries.
• The proposed algorithm shows promise for enhancing computational models of learning.
• RCHL offers improved biological plausibility compared to existing contrastive learning methods.