Time elapsed between choices in a probabilistic task correlates with repeating the same decision

Area of Science:

• Neuroscience
• Behavioral Science
• Computational Neuroscience

Background:

• Reinforcement learning (RL) principles dictate that actions followed by positive outcomes are more likely to be repeated.
• Understanding the temporal dynamics influencing decision-making is crucial for refining RL models.

Purpose of the Study:

• To investigate the impact of time elapsed since an action on subsequent choices in a rodent model.
• To evaluate how temporal factors interact with reward prediction errors in reinforcement learning.

Main Methods:

• Utilized IntelliCages to monitor behavioral choices of C57BL6/J mice over ~33 days.
• Implemented a probabilistic reward schedule where reward probabilities changed dynamically.
• Fitted behavioral data to various reinforcement learning models.

Main Results:

• Mice demonstrated increased choice repetition following rewarded actions, consistent with RL principles.
• Decision-making probability was significantly influenced by the time elapsed since the previous choice, irrespective of the outcome.
• Reinforcement learning models incorporating separate learning rates for positive/negative outcomes and a "fictitious" update best explained the data.

Conclusions:

• Temporal factors, independent of reward history, play a significant role in reinforcement learning and decision-making.
• Advanced reinforcement learning models that account for time-dependent value decay offer improved predictions of behavior.