Strongly improved stability and faster convergence of temporal sequence learning by using input correlations only.

Area of Science:

• Neuroscience
• Machine Learning
• Computational Neuroscience

Background:

• Traditional Hebbian learning correlates input and output activity to modify synaptic strength.
• This approach can lead to destabilizing autocorrelation terms, causing positive feedback and potential divergence.
• Existing solutions like weight normalization or saturation introduce other problems and limit learning rates, slowing convergence.

Purpose of the Study:

• Introduce a novel correlation-based learning rule that modifies Hebbian learning.
• Replace the output derivative with the reflex input derivative to use only input correlations (heterosynaptic learning).
• Demonstrate improved stability and significantly faster learning rates compared to existing methods.

Main Methods:

• Developed a new learning rule by replacing the output derivative with the reflex input derivative in a Hebbian-like framework.
• The new rule enforces strict heterosynaptic learning by utilizing only input correlations.
• Theoretically analyzed the rule's properties, including the removal of autocorrelation terms and the potential for high learning rates.

Main Results:

• The novel learning rule eliminates the destabilizing autocorrelation term inherent in classical Hebbian learning.
• This elimination allows for the use of high learning rates, leading to faster convergence.
• Mathematical analysis and experimental validation show that the rule can achieve one-shot learning under ideal conditions, being up to 100 times faster than previous methods.

Conclusions:

• The new heterosynaptic learning rule offers a more stable and significantly faster alternative to traditional Hebbian and ISO learning.
• It overcomes the limitations of autocorrelation and allows for rapid learning, approaching theoretical optima.
• This advancement has implications for developing more efficient and robust artificial neural networks.