Closed-form feedback-free learning with forward projection

Area of Science:

• Computational neuroscience
• Machine learning

Background:

• Current backpropagation-free methods rely on local error feedback for optimization.
• A significant limitation exists in settings lacking retrograde communication for pre-synaptic weight tuning.

Purpose of the Study:

• To introduce Forward Projection (FP), a training method that bypasses retrograde communication.
• To enable efficient neural network training using only a single forward pass.

Main Methods:

• FP employs randomized nonlinear projections to generate target pre-activation membrane potentials.
• Local loss functions are optimized via closed-form regression, eliminating the need for downstream layer feedback.
• The method utilizes a single forward pass over the dataset.

Main Results:

• FP achieves generalization performance comparable to gradient descent-based local learning methods.
• Significant training speedups are observed due to the single forward pass requirement.
• In few-shot learning, FP models demonstrate superior generalizability compared to backpropagation-optimized models.
• Layer-wise membrane potentials in FP networks provide interpretable label predictions.

Conclusions:

• Forward Projection offers an efficient and interpretable alternative to traditional training methods.
• FP is particularly advantageous in scenarios with limited or no retrograde communication.
• The method shows promise for biomedical applications, including identifying diagnostic features in few-shot learning tasks.