Signal Propagation: The Framework for Learning and Inference in a Forward Pass

Area of Science:

• Artificial Intelligence
• Computational Neuroscience
• Machine Learning

Background:

• Backpropagation (BP) is the standard for neural network training but has limitations.
• BP requires specific network structures like feedback connectivity and a backward pass.
• Biological and hardware implementations of BP face significant constraints.

Purpose of the Study:

• Introduce signal propagation (sigprop) as a forward-pass-only learning framework.
• Provide an alternative to backpropagation that is more biologically and hardware-compatible.
• Demonstrate sigprop's efficiency and effectiveness.

Main Methods:

• Developed the sigprop learning framework utilizing only a forward pass for signal propagation and parameter updates.
• Applied sigprop to train continuous-time neural networks with Hebbian updates.
• Trained spiking neural networks (SNNs) using sigprop with voltage-based or surrogate gradient methods.

Main Results:

• Sigprop enables global supervised learning without backward connectivity or other BP constraints.
• Sigprop demonstrates greater efficiency in terms of time and memory compared to BP-based approaches.
• Sigprop provides useful learning signals comparable to BP and supports biologically plausible learning mechanisms.

Conclusions:

• Sigprop presents a viable, efficient, and flexible alternative to backpropagation for neural network training.
• The framework aligns well with biological learning principles and hardware implementation possibilities.
• Sigprop facilitates parallel training and offers a new direction for artificial intelligence and neuroscience research.