Adaptive reinforcement learning is causally supported by anterior cingulate cortex and striatum

Area of Science:

• Neuroscience
• Cognitive Science
• Computational Neuroscience

Background:

• Reinforcement learning benefits from adaptive strategies.
• These strategies involve adjusting exploration-exploitation, working memory, and attention.
• The anterior cingulate cortex (ACC) and striatum are key brain regions involved.

Purpose of the Study:

• Investigate how ACC and striatum support adaptive learning.
• Examine roles during feature-based attention learning with varying uncertainty and saliency.
• Determine the impact of direct brain stimulation on these processes.

Main Methods:

• Gaze-contingent electrical stimulation of ACC and striatum.
• Behavioral testing of adaptive reinforcement learning.
• Computational modeling to interpret neural mechanisms.
• Analysis of neuronal selectivity in ACC and striatum.

Main Results:

• ACC stimulation impaired learning at high uncertainty; striatum stimulation improved it.
• Modeling indicated ACC stimulation hindered exploration optimization and prediction error use.
• Striatum stimulation enhanced the updating of value expectations.
• Neuronal activity in ACC tracked error history and uncertainty; striatal neurons favored certain, high-value choices.

Conclusions:

• ACC and striatum play distinct, opposing roles in adaptive reinforcement learning.
• These regions optimize the guidance of exploration toward reward-relevant cues during uncertainty.
• Findings provide insights into neural mechanisms of attention and learning.