Opponent Learning with Different Representations in the Cortico-Basal Ganglia Circuits.

Area of Science:

• Computational neuroscience
• Reinforcement learning
• Neurobiology

Background:

• Basal ganglia (BG) pathways are theorized to learn from positive and negative feedback.
• Differential cortical inputs suggest distinct state/action representations in BG pathways.
• Investigating computational benefits of combined representations and learning rates.

Purpose of the Study:

• To explore computational benefits of distinct state representations (individual representation [IR] vs. successor representation [SR]) within basal ganglia pathways.
• To examine the impact of varying learning rates for positive and negative reward prediction errors (RPEs) on agent performance.

Main Methods:

• Modeled an agent with two learning systems, each using IR or SR.
• Varied the combination of IR/SR and learning rates for positive/negative RPEs.
• Evaluated agent performance in a dynamic reward navigation task.

Main Results:

• A combination of SR-based learning for positive RPEs and IR-based learning for negative RPEs yielded optimal performance.
• This dual system exhibited comparable, oppositely signed activities, aligning with BG pathway profiles.
• The model offers a unified explanation for cortico-BG circuit findings.

Conclusions:

• Combining distinct representations (SR for appetitive, IR for aversive learning) is an effective strategy in dynamic environments.
• This learning strategy may be implemented within cortico-basal ganglia circuits.
• Provides novel insights into the functional significance and mechanisms of cortico-BG interactions.