An anatomical substrate of credit assignment in reinforcement learning

Area of Science:

• Neuroscience
• Computational Neuroscience
• Artificial Intelligence

Background:

• Credit assignment is a fundamental challenge in biological and artificial learning systems.
• Current artificial intelligence relies on backpropagation, lacking a clear biological correlate.
• The basal ganglia are implicated in learning, but mechanisms for credit assignment remain unclear.

Purpose of the Study:

• To investigate the synaptic architecture of songbird basal ganglia (Area X) for local credit assignment.
• To test predictions of a node perturbation model of reinforcement learning in a biological system.
• To develop a biophysical model of reinforcement learning based on detailed neural data.

Main Methods:

• Automated connectomic analysis using two volume electron microscopy (vEM) datasets.
• Comparison of axonal termination patterns with model predictions.
• Construction of a biophysical model of reinforcement learning.

Main Results:

• Synaptic architecture in Area X supports local credit assignment via a node perturbation algorithm variant.
• Axons encoding exploratory variability terminate on dendritic shafts.
• Axons encoding song timing (context) terminate on dendritic spines.
• A biophysical model demonstrates efficient learning facilitated by this synaptic dichotomy.

Conclusions:

• The findings provide strong evidence for a biologically plausible credit assignment model in vertebrate basal ganglia.
• This model offers a potential mechanism for learning that differs from artificial intelligence backpropagation.
• The study highlights the role of synaptic architecture in enabling complex computational functions in the brain.