Nash equilibria in human sensorimotor interactions explained by Q-learning with intrinsic costs.

Area of Science:

• Neuroscience
• Cognitive Science
• Computational Neuroscience

Background:

• The Nash equilibrium concept is crucial for understanding human sensorimotor interactions in multi-agent tasks.
• The mechanisms by which humans reach these equilibria during continuous motor tasks remain unclear.

Purpose of the Study:

• To compare different reinforcement learning models against human behavior in sensorimotor interactions.
• To investigate how learning dynamics influence the selection of Nash equilibrium solutions.

Main Methods:

• Human participants engaged in sensorimotor tasks with haptic feedback, modeled after classic games (Prisoner's Dilemma, Matching Pennies).
• Continuous formulations of reinforcement learning algorithms and game-theoretic solutions were employed for detailed analysis.
• Comparison of discrete vs. continuous analysis approaches to differentiate learning algorithms.

Main Results:

• Discrete analysis of sensorimotor interactions into binary choices failed to distinguish between learning algorithms.
• Continuous analysis revealed distinct predictions based on different learning algorithms.
• Q-learning with intrinsic costs, which penalize deviations from average behavior, best explained the observed human data.

Conclusions:

• Understanding sensorimotor interactions requires studying diverse learning algorithms, not solely focusing on the Nash equilibrium concept.
• Game-theoretic analysis alone is insufficient as learning dynamics impose preferences on equilibrium selection.
• Q-learning with intrinsic costs provides a strong model for explaining human behavior in these interactive tasks.