Actor-critic networks with analogue memristors mimicking reward-based learning

Area of Science:

• Neuromorphic Engineering
• Computational Neuroscience

Background:

• Current bio-inspired computing often mimics only parts of the brain.
• Memristive hardware is typically used for limited functions in learning algorithms, requiring software for complex tasks.

Purpose of the Study:

• To demonstrate a fully integrated reinforcement learning system using analogue memristors.
• To create a bio-inspired neural network architecture that performs reward-based learning entirely in hardware.

Main Methods:

• Implemented an actor-critic temporal difference algorithm on analogue memristors.
• Utilized memristors as multi-purpose elements: synaptic weights, weight update calculators, and action determiners.
• Tested the framework on T-maze and Morris water maze navigation tasks.

Main Results:

• Achieved online weight training and direct hardware calculation of temporal difference errors.
• Enabled complete in-memory computation, eliminating data movement for weight training.
• Successfully demonstrated navigation in simulated environments using the memristor-based system.

Conclusions:

• This work presents the first fully in-memory, online reinforcement learning system for bio-inspired computing.
• The approach paves the way for more efficient and brain-like neuromorphic computing engines.
• Analogue memristors can serve as versatile components for advanced artificial intelligence hardware.