Frequency effects in action versus value learning.

Area of Science:

• Cognitive Psychology
• Reinforcement Learning
• Decision Making

Background:

• Reinforcement learning research indicates a frequency effect: preference for lower-reward options experienced more often.
• This challenges the assumption that expected value relies solely on average reward, suggesting cumulative instances are key.
• Option frequency during training might influence instrumental action reinforcement, impacting subsequent choices.

Purpose of the Study:

• To investigate the roles of action reinforcement and option value in the frequency effect.
• To differentiate between value-based and action-reinforcement-based contributions to choice preference.

Main Methods:

• Two experiments were conducted, varying the training method for choice options.
• Experiment 1 used standard choice training; Experiment 2 used individual option presentation to isolate action reinforcement.
• A reward likelihood rating task was included in both experiments to assess option value independent of action reinforcement.

Main Results:

• Reducing action reinforcement (Experiment 2) weakened the frequency effect compared to standard training (Experiment 1).
• Frequency effects persisted even when action reinforcement was minimized, indicating a role for option value.
• Reward likelihood ratings mirrored choice preferences, suggesting action reinforcement may also bias value judgments.

Conclusions:

• Action reinforcement contributes to the frequency effect in reinforcement learning.
• Option value, influenced by cumulative experience, also plays a significant role.
• The findings suggest a complex interplay where reinforcement history can shape perceived option value.