Spatiotemporal Decoding of Explore-Exploit Decisions in the Human Brain

Area of Science:

• Neuroscience
• Cognitive Science
• Computational Psychiatry

Background:

• Adaptive behavior necessitates balancing exploitation of known rewards with exploration of novel opportunities.
• This explore-exploit decision-making is managed by a distributed neural network, primarily involving the frontopolar cortex (FPC) and ventromedial prefrontal cortex (vmPFC).
• Understanding the temporal dynamics of these neural computations is crucial for elucidating how the brain navigates this fundamental tradeoff.

Purpose of the Study:

• To precisely map the spatiotemporal emergence of explore-exploit decisions across cortical networks.
• To investigate the neural mechanisms underlying the brain's management of the explore-exploit dilemma using advanced computational modeling and neuroimaging.

Main Methods:

• Combined magnetoencephalography (MEG) with partially observable Markov decision process (POMDP) modeling during a reinforcement learning task.
• Decoded POMDP-derived choice policies in MEG source space to track neural activity.
• Analyzed millisecond-resolved neural data to identify the timing and location of explore-exploit computations.

Main Results:

• Explore-exploit policy implementation shows a hierarchical functional dissociation across the rostral prefrontal cortex.
• The lateral FPC initiates the strategic shift towards exploration significantly earlier (hundreds of milliseconds before choice execution) than the vmPFC and orbitofrontal cortex (OFC).
• vmPFC and OFC exhibit a delayed, transient neural response before choice execution on exploration trials, followed by sustained representation of empirical value post-feedback for updating reward expectancies.

Conclusions:

• The human brain resolves the explore-exploit dilemma through a temporally organized neural process.
• Early strategic initiation of exploration by the FPC precedes delayed responses in the vmPFC and OFC.
• Sustained outcome evaluation in vmPFC and OFC optimizes future actions, demonstrating a sophisticated neural strategy for adaptive decision-making.