Prefrontal solution to the bias-variance tradeoff during reinforcement learning

Area of Science:

• Neuroscience
• Computational Neuroscience
• Cognitive Science

Background:

• Accumulating evidence suggests the brain integrates diverse value learning strategies to minimize prediction error.
• The bias-variance tradeoff presents a significant challenge for value learning, as it imposes distinct performance constraints on different learning strategies.

Purpose of the Study:

• To investigate how the brain adaptively resolves the bias-variance tradeoff during reinforcement learning.
• To explore the role of prediction error baseline correction in mitigating irreducible error effects on value learning.
• To identify neural correlates of adaptive bias-variance tradeoff resolution.

Main Methods:

• Behavioral experiments utilizing a Markov decision task with dynamic context changes.
• Analysis of behavioral data to identify adaptive control mechanisms.
• Neuroimaging techniques to identify neural representations of prediction error baseline in key prefrontal cortex regions.

Main Results:

• Behavioral evidence demonstrates adaptive control in response to context changes.
• The prediction error baseline appears to signal environmental shifts, enhancing learning adaptability.
• Multiplexed representations of prediction error baseline were identified in the ventrolateral and ventromedial prefrontal cortex.

Conclusions:

• The brain adaptively resolves the bias-variance tradeoff during reinforcement learning.
• Baseline correction of prediction error is a critical mechanism for optimizing value learning.
• The ventrolateral and ventromedial prefrontal cortex play crucial roles in this adaptive process, integrating model-based and model-free reinforcement learning signals.