Decision making under uncertainty: a neural model based on partially observable markov decision processes.

Area of Science:

• Neuroscience
• Computational Neuroscience
• Cognitive Science

Background:

• Animals face challenges in selecting actions with noisy sensory data and incomplete world knowledge.
• While Bayesian inference is linked to perception, its role in action selection remains unclear.

Purpose of the Study:

• To propose a neural model for action selection and decision-making based on partially observable Markov decision processes (POMDPs).
• To unify explanations for experimental findings in decision-making involving information gathering and overt actions.

Main Methods:

• Developed a neural model using POMDP theory and temporal difference (TD) learning for reward maximization.
• Modeled action selection based on posterior state distributions (belief states) rather than single optimal estimates.
• Investigated neural architecture involving neocortex and basal ganglia roles in belief computation, representation, value computation, and action selection.

Main Results:

• Model neurons show belief responses similar to primate neocortical LIP neurons in motion discrimination tasks.
• The model naturally derives thresholds for switching between information gathering and action during reward maximization.
• Reward prediction error in the model aligns with basal ganglia dopaminergic responses.
• The model predicts a collapsing decision threshold for time-sensitive tasks, consistent with experimental data.

Conclusions:

• The proposed model offers a novel framework for understanding neural decision-making.
• It highlights the crucial role of neocortex-basal ganglia interactions in linking probabilistic representations to reward-maximizing actions.