Signatures of Perseveration and Heuristic-Based Directed Exploration in Two-Step Sequential Decision Task Behaviour

Area of Science:

• Cognitive Neuroscience
• Computational Psychiatry
• Reinforcement Learning Theory

Background:

• Reinforcement Learning (RL) processes like model-based (MB) control and exploration are crucial in neuroscience and psychiatry.
• Dysregulation in these RL processes is linked to psychiatric disorders, but they are often studied in isolation.
• Standard hybrid models of the two-step task (TST) are used to measure MB control.

Purpose of the Study:

• To extend standard hybrid models of the TST to quantify exploration and perseveration mechanisms.
• To compare different computational model extensions for the TST.
• To investigate the neurocomputational underpinnings of decision-making in psychiatric contexts.

Main Methods:

• Implemented and compared various computational model extensions for a sequential RL task (two-step task).
• Utilized two independent datasets from different task variants.
• Employed posterior predictive checks to validate model performance.

Main Results:

• An extended hybrid RL model incorporating higher-order perseveration and heuristic-based exploration provided the best fit to the data.
• A simpler model with complex perseveration alone was also a good fit.
• A significant positive effect of directed exploration on stage-one choice probabilities was identified.

Conclusions:

• The extended RL model successfully reproduced choice patterns across both datasets.
• Findings highlight the importance of considering combined exploration and perseveration mechanisms.
• Results have implications for computational psychiatry and identifying neurocognitive endophenotypes.