A reinforcement learning model with choice traces for a progressive ratio schedule

Area of Science:

• Neuroscience
• Behavioral Science
• Computational Psychiatry

Background:

• The progressive ratio (PR) lever-press task is a standard measure of motivation.
• Current PR tasks provide only a single breakpoint measure, limiting analysis of motivational changes.
• Variations in lever pressing and magazine nosepoking during PR tasks are not well-captured.

Purpose of the Study:

• To develop advanced reinforcement learning models for analyzing choice behavior in PR tasks.
• To identify a model that can capture motivational dynamics beyond the breakpoint.
• To investigate the neural correlates of motivation using dopamine measurements.

Main Methods:

• Constructed four reinforcement learning models, including Q-learning and perseverance models.
• Incorporated reinforced lever presses, non-reinforced lever presses, and magazine nosepokes into models.
• Used fiber photometry with a dopamine sensor (GRAB_DA2m) to measure dopamine in male mice during PR tasks.
• Administered methamphetamine (METH) to assess model's ability to capture drug effects.

Main Results:

• The perseverance model best predicted behavioral changes, including a reduction in reward prediction errors (RPEs) after magazine nosepokes.
• Dopamine dynamics in the ventral striatum correlated with the model's predictions.
• Methamphetamine increased magazine nosepokes without altering the breakpoint, a modulation captured by the perseverance model.
• The model highlighted the role of initial action values in behavioral modulation.

Conclusions:

• The perseverance model offers a more nuanced understanding of motivation and choice behavior in PR tasks.
• This model can reveal the impact of psychoactive drugs on motivational processes.
• The findings support the use of computational models for analyzing complex behaviors and neural signals.