The representation and valuation of subgoals in the human brain during model-based hierarchical behavior

Planning and performing complex, sequential behavior toward distant goals relies on dividing behavior into structured segments. This hierarchical organization requires the brain to designate certain states as subgoals to mark successful segment completion. How the brain represents subgoals and computes decision values as a function of them remains unknown. While most models of hierarchical behavior lack environmental knowledge, decision-making involves planning with an internal world model. Consequently, how the brain integrates hierarchical and model-based processes has yet to be explained. Using a sequential-subgoal decision-making task with functional magnetic resonance imaging (fMRI), we evoked hierarchical, model-based behavior. We decoded the current subgoal in insula and ventromedial prefrontal cortex activity-a critical latent representation for orienting sequential behavior. Using a model-based, hierarchical reinforcement learning model, we identified key decision value signals in the frontal cortex. These findings illuminate neural correlates of subgoals and decision values computed as a function of subgoals and environmental knowledge.