Learning reward frequency over reward probability: A tale of two learning rules

Area of Science:

• Cognitive Science
• Neuroscience
• Decision Science

Background:

• Adaptive behavior relies on learning expected reward values of choices.
• Dominant value learning models, like the delta rule, may not fully capture how presentation frequency influences choice preferences.

Purpose of the Study:

• To investigate whether reward learning is driven by reward probability or frequency of reward.
• To compare the predictive accuracy of average reward (delta model) versus cumulative reward (decay model) in accounting for choice preferences.

Main Methods:

• Participants engaged in a binary-outcome choice task with options of varying reward probabilities.
• A key manipulation involved differential trial frequencies for option pairs, creating a dissociation between average and cumulative reward.
• Model comparison was performed between the delta and decay models, alongside simpler and more complex variants.

Main Results:

• A majority of participants preferred the option associated with higher cumulative reward, even when another option offered a higher average reward.
• This preference pattern was accurately predicted by the decay model but not the delta model.
• Models incorporating cumulative reward learning provided a superior fit to the observed choice data.

Conclusions:

• Human reward learning and choice behavior are significantly influenced by cumulative reward exposure.
• Models emphasizing cumulative reward learning offer a more robust explanation of decision-making compared to those solely based on average reward.