Policy complexity suppresses dopamine responses.

Area of Science:

• Neuroscience
• Cognitive Science
• Computational Biology

Background:

• Information processing capacity is a fundamental constraint on cognitive and behavioral performance.
• Understanding how biological systems optimize performance under capacity limits is crucial for neuroscience and artificial intelligence.

Purpose of the Study:

• To investigate how animals achieve near-optimal performance on perceptual decision tasks despite inherent information processing limitations.
• To explore the role of policy complexity and reinforcement learning mechanisms, particularly dopaminergic signaling, in this optimization.

Main Methods:

• Quantified animal performance on a perceptual decision task.
• Measured policy complexity using mutual information between states and actions.
• Investigated the relationship between policy complexity, dopamine responses to reward, and behavioral sensitivity.

Main Results:

• Animal performance was near-optimal given their measured policy complexity.
• Higher policy complexity was associated with suppressed midbrain dopamine responses to rewards.
• Policy compression reduced behavioral sensitivity to reward outcomes.

Conclusions:

• Reinforcement learning with a penalty on high complexity policies, modulated by dopamine, can explain the observed behavioral profile.
• Policy compression is a key factor shaping fundamental reinforcement learning mechanisms in the brain.
• Dopaminergic signals play a critical role in balancing performance and computational efficiency.