Equilibrium Propagation: Bridging the Gap between Energy-Based Models and Backpropagation.

Area of Science:

• Computational Neuroscience
• Machine Learning
• Artificial Intelligence

Background:

• Energy-based models (EBMs) are a class of models used in machine learning and neuroscience.
• Training EBMs often relies on algorithms like Backpropagation, which have limitations in biological plausibility.
• Existing learning rules like Contrastive Hebbian Learning and Contrastive Divergence face theoretical challenges.

Purpose of the Study:

• To introduce Equilibrium Propagation (EP), a new learning framework for EBMs.
• To present a biologically plausible alternative to Backpropagation for training neural networks.
• To address theoretical limitations of existing learning algorithms for EBMs.

Main Methods:

• Equilibrium Propagation utilizes a single type of neural computation for both prediction and error propagation phases.
• The framework involves nudging predictions towards configurations that minimize prediction error.
• In recurrent networks, perturbations at the output layer implicitly propagate backward through hidden layers.

Main Results:

• EP computes the gradient of a well-defined objective function, overcoming theoretical issues of prior methods.
• The implicit error propagation in EP mirrors the back-propagation of error derivatives.
• EP successfully trained multi-layer recurrent networks on the permutation-invariant MNIST task.

Conclusions:

• Equilibrium Propagation offers a unified and biologically plausible learning mechanism for EBMs.
• The framework demonstrates that inference and error back-propagation can be achieved using the same neural computation.
• EP provides a viable alternative to Backpropagation, potentially bridging the gap between artificial and biological learning.