Combined model-free and model-sensitive reinforcement learning in non-human primates

Area of Science:

• Neuroscience
• Cognitive Science
• Computational Neuroscience

Background:

• Reinforcement learning (RL) theory distinguishes between model-free (MF) and model-sensitive (MS) strategies.
• MF strategies rely on reward history, while MS strategies utilize task dynamics knowledge.
• Combining MF and MS strategies offers complementary computational strengths, but empirical evidence, especially in non-human primates, is limited.

Purpose of the Study:

• To investigate the interplay of MF and MS reinforcement learning strategies in rhesus monkeys.
• To discriminate between MF and MS strategy use in a novel two-stage decision task.
• To analyze how task structure and reward history influence choice behavior and response vigor.

Main Methods:

• Rhesus monkeys were trained on a two-stage decision task.
• Descriptive analysis of choice behavior and response vigor was performed.
• Trial-by-trial computational modeling was used to analyze decision-making strategies, including a novel combined RL model.

Main Results:

• Both task structure and reward history significantly influenced choice and response vigor.
• Behavior was best explained by a combination of MF and MS strategies, with a dominant and persistent model-sensitive influence.
• A new RL model incorporating specific credit assignment weighting was developed to account for observed behavior.
• Response vigor showed distinct MF and MS influences.

Conclusions:

• Rhesus monkeys employ a sophisticated combination of reinforcement learning strategies.
• Model-sensitive strategies play a dominant role in decision-making for this species.
• These findings offer novel insights into the neural and computational mechanisms of reinforcement learning in primates.