Information bottleneck-based Hebbian learning rule naturally ties working memory and synaptic updates

Area of Science:

• Computational neuroscience
• Artificial intelligence
• Machine learning

Background:

• Deep neural networks (DNNs) face significant energy costs for training and deployment.
• Spiking neural networks (SNNs) offer an energy-efficient alternative on neuromorphic hardware but face training challenges.
• Back-propagation, a standard DNN training method, is biologically implausible for SNNs.

Purpose of the Study:

• To develop a biologically plausible and energy-efficient training method for SNNs.
• To address the limitations of existing SNN training rules, particularly the need for concurrent sample processing.
• To establish a direct link between working memory and synaptic plasticity in SNNs.

Main Methods:

• Proposed a novel three-factor Hebbian update rule for SNNs.
• Incorporated an auxiliary memory network to handle global error signals across samples.
• Trained the auxiliary network independently prior to the primary network.
• Evaluated performance on image classification tasks.

Main Results:

• Achieved performance comparable to baseline methods on image classification.
• Demonstrated a direct connection between working memory and synaptic updates, unlike back-propagation.
• Showcased the impact of memory capacity on learning performance.

Conclusions:

• The proposed learning rule offers a biologically plausible and efficient method for training SNNs.
• This work establishes an explicit link between working memory and synaptic plasticity.
• Suggests a new perspective on neural computation where layers balance memory-informed compression and task performance.