Model-based and model-free valuation signals in the human brain vary markedly in relation to individual differences...

Human action selection under reinforcement is believed to rely on two distinct strategies: model-free and model-based reinforcement learning. While behavior in sequential decision-making tasks often reflects a mixture of both, the neural basis of individual differences in their expression remains unclear. Here, we conduct a large-scale fMRI study with 179 participants performing a variant of the two-step task. Using both cluster-defined subgroups and computational parameter estimates, we find that in the ventromedial prefrontal cortex, model-based value signals are strongly linked to the degree of model-based behavioral reliance, whereas model-free signals are ubiquitous across individuals regardless of model-free behavioral influence. Individuals lacking model-based behaviors and model-based neural signals, exhibit impaired state prediction errors (a key signal for learning a model of the environment) in the dorsolateral prefrontal cortex and intraparietal sulcus, suggesting that reduced model-based control may depend in part on underlying difficulties in forming accurate model-based predictions.