People Can Adaptively Exploit Model-free and Model-based Reinforcement Learning in Competitive Games.

Area of Science:

• Behavioral economics
• Computational neuroscience
• Game theory

Background:

• Competitive social interactions drive strategy learning.
• Limited research exists on how opponent predictability influences human strategy and performance over time.

Purpose of the Study:

• To investigate how human strategic behavior and performance change based on opponent predictability.
• To compare human strategies against model-free and model-based reinforcement learning (RL) algorithms in Rock, Paper, Scissors.

Main Methods:

• Two experiments were conducted using Rock, Paper, Scissors with human participants playing against RL-programmed computer opponents.
• Experiment 1 featured model-free RL algorithms, updating only selected action values.
• Experiment 2 utilized model-based RL algorithms, updating both selected and unselected action values.

Main Results:

• Participants consistently outperformed both model-free and model-based RL opponents.
• Human participants employed distinct strategies: Win-Stay/Lose-Shift in Experiment 1 and Win-Shift/Lose-Shift in Experiment 2.
• Observed human strategies diverged from predictions of standard RL models and learning theories.

Conclusions:

• Human strategic decision-making in competitive environments is more dynamic and adaptive than predicted by conventional RL models.
• Reinforcement learning serves as a framework for understanding strategic decision-making, but human adaptability requires further investigation.
• Future research should explore additional computational models and competitive games to understand adaptive strategy shifts.