Adaptive Reinforcement Learning is causally supported by Anterior Cingulate Cortex and Striatum

Area of Science:

• Neuroscience
• Cognitive Science
• Decision Making

Background:

• The anterior cingulate cortex (ACC) is implicated in determining object relevance in dynamic environments.
• It is hypothesized that the ACC integrates reward information to guide exploration versus avoidance decisions.

Purpose of the Study:

• To investigate the causal role of the ACC in adaptive behavior, particularly in inferring unknown reward values through exploration.
• To examine how ACC function is affected by feature uncertainty and motivational value during learning.

Main Methods:

• Nonhuman primates performed a learning task with manipulated feature uncertainty and motivational value.
• Transient microstimulation of the ACC was applied during stimulus foveation.
• Adaptive reinforcement learning models were used to analyze behavior.

Main Results:

• ACC stimulation impaired learning selectively under high feature uncertainty and motivational value.
• This impairment was linked to deficits in feature-specific credit assignment and weighting of prediction errors.
• A meta-learning impairment in adaptively increasing exploration and impaired working memory for object values were observed.

Conclusions:

• The ACC causally contributes to adaptive behavior by meta-adjusting the exploration-exploitation balance.
• Reward history traces in the ACC are essential for modulating working memory strength and guiding exploration in dynamic environments.