Reinforcement Learning: Full Glass or Empty - Depends Who You Ask

Area of Science:

• Neuroscience
• Computational Neuroscience
• Artificial Intelligence

Background:

• The prediction error theory of dopamine is a cornerstone in understanding reward-based learning.
• Existing models primarily focus on the average expected reward, which may not fully capture complex neural responses.

Purpose of the Study:

• To extend the prediction error theory of dopamine by incorporating the full distribution of future rewards.
• To investigate if this extended theory provides a more accurate account of dopamine responses compared to traditional models.

Main Methods:

• Importing concepts from artificial intelligence, specifically distributional reinforcement learning.
• Developing a computational model that represents the entire probability distribution of future rewards.

Main Results:

• The extended theory, by considering the full reward distribution, offers a more comprehensive explanation for dopamine neuron activity.
• This distributional approach better accounts for empirical observations of dopamine signaling under various reward contingencies.

Conclusions:

• Representing the full distribution of future rewards offers a more nuanced and accurate framework for understanding dopamine's role in learning and decision-making.
• This work highlights the potential of integrating advanced artificial intelligence concepts into neuroscience for improved theoretical models.