Neural systems for choice and valuation with counterfactual learning signals

Area of Science:

• Neuroscience
• Computational Psychiatry
• Decision Science

Background:

• Reinforcement learning models are essential for understanding decision-making.
• Fictive prediction error (FPE) signals, representing counterfactual outcomes, may enhance these models.
• Neurotransmitter systems like serotonin and dopamine play critical roles in learning and valuation.

Purpose of the Study:

• To test a computational model of reinforcement learning with and without FPE signals.
• To investigate the role of counterfactual consequences in action-specific expected value representation.
• To determine the neuroanatomical and neuromodulatory underpinnings of FPE-based valuation.

Main Methods:

• 80 male participants underwent dietary depletion of tryptophan or tyrosine/phenylalanine to manipulate serotonin and dopamine.
• Participants completed a strategic sequential investment task while undergoing fMRI.
• A modified Q-learning model incorporating FPE was compared to a standard Q-learning model.

Main Results:

• The FPE model provided a significantly better fit to participant data than the standard model.
• Expected value from the FPE model correlated with BOLD signals in vmPFC and OFC.
• Dietary manipulations revealed differential neural responses in vmPFC, caudate, and SN, modulated by neurotransmitters.

Conclusions:

• FPE signals are a critical component of valuation in decision-making.
• The neural representation of expected value involves interactions between cortical and subcortical structures.
• Serotonergic and dopaminergic systems modulate the neural processing of expected value through FPE signals.