Brain-inspired synaptic transistors for in-situ spiking reinforcement learning with eligibility trace

Area of Science:

• Materials Science
• Neuroscience
• Computer Science

Background:

• Current artificial neural networks lack biological mechanisms crucial for advanced reinforcement learning (RL).
• Emerging materials offer potential for mimicking complex RL dynamics.
• Spiking neural networks (SNNs) show promise for brain-inspired computing.

Purpose of the Study:

• To develop a brain-inspired SNN-based RL computing architecture.
• To utilize α-In2Se3 ferroelectric semiconductor field-effect transistors for RL.
• To implement biological learning mechanisms like eligibility traces and dynamic reward signaling.

Main Methods:

• Fabrication and characterization of α-In2Se3 ferroelectric semiconductor field-effect transistors.
• Design of an RL neural network using an α-In2Se3 transistor array.
• Demonstration of autonomous driving tasks using the developed architecture.

Main Results:

• The α-In2Se3 transistor enabled reward signal modulation and eligibility trace decay.
• The architecture performed in-situ reward-based weight updates and eligibility trace decay.
• Autonomous driving tasks were successfully demonstrated with the RL neural network.

Conclusions:

• The developed architecture enables fully functional, energy-efficient, and low-overhead spiking-based reinforcement learning.
• This approach integrates essential biological learning mechanisms into hardware.
• The study paves the way for advanced artificial general intelligence.