Distributed representations of temporally accumulated reward prediction errors in the mouse cortex

Area of Science:

• Neuroscience
• Computational Neuroscience
• Reinforcement Learning

Background:

• Reward prediction errors (RPEs) are crucial for learning, but their neural implementation in the brain remains unclear.
• Reinforcement learning (RL) theory suggests accumulated RPEs enhance learning efficiency.

Purpose of the Study:

• To investigate whether the brain utilizes mechanisms similar to RL for processing RPEs.
• To identify neural populations encoding RPE accumulation in the mouse cortex.

Main Methods:

• Constructed RL-based theoretical models.
• Utilized multiregional two-photon calcium imaging in the mouse dorsal cortex.
• Analyzed neural activity related to RPE accumulation and reward function manipulations.

Main Results:

• Identified a neuronal population modulated by RPE accumulation.
• Observed sequential activation of RPE-encoding neurons within trials, forming distributed assemblies.
• Found RPE representations aligned with RL predictions, emerging during learning.
• Revealed region-specific encoding, with higher-order cortical regions showing long-term RPE accumulation encoding.

Conclusions:

• Cortical RPE computation involves a complex, distributed neural code.
• This neural mechanism potentially enhances learning efficiency in animals by integrating RPE signals over time.