Advancing the forward-forward algorithm towards high-performance deep local learning.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computational Neuroscience

Background:

• Backpropagation (BP) faces limitations in memory efficiency and parallelization.
• Forward-Forward (FF) algorithm offers a local learning alternative but struggles with performance and generalization.
• Existing FF methods lack effective learning strategies for complex tasks.

Purpose of the Study:

• To improve the performance and generalization of the FF algorithm in supervised learning.
• To preserve the advantageous local computational properties of FF.
• To extend FF-based local learning to spiking neural networks (SNNs) and neuromorphic hardware.

Main Methods:

• Reformulated FF using distance metric learning, proposing a distance-forward (DF) algorithm.
• Developed a goodness-based N-pair margin loss for discriminative feature learning.
• Integrated layer-collaboration local update strategies to mitigate information loss.
• Extended DF to SNNs with a goodness function for temporal spike sequences.

Main Results:

• DF algorithm surpasses existing FF models and local learning approaches across eight datasets.
• DF methods achieve over 60% memory cost reduction compared to BP training.
• Proposed methods demonstrate enhanced robustness against hardware-related noise.
• Effective implementation for event-driven processing on neuromorphic hardware.

Conclusions:

• The proposed DF algorithm offers an efficient and robust local learning solution.
• DF enhances FF performance and generalization while maintaining memory efficiency.
• The approach shows promise for future FF algorithm designs and many-core hardware applications.