Scaling prediction errors to reward variability benefits error-driven learning in humans.

Area of Science:

• Cognitive Science
• Neuroscience
• Reinforcement Learning

Background:

• Effective error-driven learning necessitates adaptation to environmental reward variability.
• Previous research indicated learning rate decay as a key adaptive mechanism.
• The role of reward prediction error (RPE) scaling in this adaptation remained less understood.

Purpose of the Study:

• To investigate the influence of RPE scaling on learning performance.
• To examine how individuals adjust learning based on varying reward probabilities and standard deviations.
• To determine the relationship between RPE scaling and accuracy in predicting reward means.

Main Methods:

• Participants explicitly predicted reward magnitudes from distributions with varying standard deviations.
• Reinforcement learning models were fitted to participant data.
• Learning rate decay and RPE scaling were analyzed in relation to prediction accuracy.

Main Results:

• Data analysis revealed significant scaling of RPEs, alongside previously observed learning rate decay.
• RPE scaling strongly correlated with individual learning performance (prediction accuracy).
• Participants scaling RPEs relative to standard deviation showed consistent performance across different variability levels, unlike those with exaggerated scaling.

Conclusions:

• RPE scaling is a crucial adaptive mechanism in error-driven learning, complementing learning rate decay.
• Efficient RPE scaling enhances learning robustness to environmental reward variability.
• Exaggerated RPE scaling can lead to impaired learning performance, highlighting the importance of appropriate adaptation.