A comparison of reinforcement learning models of human spatial navigation

Area of Science:

• Cognitive Science
• Computational Neuroscience
• Human Navigation

Background:

• Reinforcement learning (RL) models are widely used for decision-making but less explored in human spatial navigation.
• Understanding individual differences in navigation requires quantitative methods to assess strategies and their consistency.
• Few studies systematically compare RL models across varying navigation demands.

Purpose of the Study:

• To apply and compare five RL models (model-free, model-based, hybrid) to human spatial navigation behavior.
• To investigate how navigation requirements influence the interplay between navigation strategy and its consistent application.
• To quantitatively characterize individual differences in human wayfinding strategies.

Main Methods:

• One hundred and fourteen participants performed wayfinding tasks in a virtual environment with manipulated navigation requirements.
• Five reinforcement learning models were fitted to participant navigation data across different task phases.
• Correlations were analyzed between model parameters (strategy) and navigation behavior (consistency).

Main Results:

• A hybrid reinforcement learning model best explained navigation behavior across all tested requirements.
• Participants predominantly use a combination of model-free (route) and model-based (mapping) learning.
• The weight on model-based learning correlated with exploration/exploitation tendencies, modulated by task demands.

Conclusions:

• Human spatial navigation relies on a flexible blend of learning strategies, best captured by a hybrid RL model.
• The relationship between an individual's navigation strategy and their consistency in applying it is dynamic and task-dependent.
• RL models provide a powerful framework for understanding individual variations in human navigation.