Combining backpropagation with Equilibrium Propagation to improve an Actor-Critic reinforcement learning framework.

Area of Science:

• Computational Neuroscience
• Machine Learning
• Artificial Intelligence

Background:

• Backpropagation (BP) is a widely used neural network training algorithm but faces questions regarding its biological plausibility.
• Equilibrium Propagation (EP) has emerged as a more biologically plausible alternative, demonstrating efficacy in tasks like image classification.

Purpose of the Study:

• To introduce the first Actor-Critic reinforcement learning architecture trained using Equilibrium Propagation.
• To evaluate the performance of EP in reinforcement learning and compare it to BP.

Main Methods:

• Developed an Actor-Critic architecture where the actor network is trained via EP.
• Tested the model on standard control tasks used for benchmarking BP-based reinforcement learning agents.

Main Results:

• The EP-based Actor-Critic model successfully solved basic control tasks.
• The trained model exhibited more consistent high-reward behavior compared to a model trained solely with BP.

Conclusions:

• Equilibrium Propagation is a viable and potentially superior training mechanism for reinforcement learning.
• EP-based models may offer more stable and consistent performance in reinforcement learning applications.