Reinforcement learning approaches to hippocampus-dependent flexible spatial navigation.

Area of Science:

• Neuroscience
• Computational Neuroscience
• Cognitive Science

Background:

• Spatial navigation is a fundamental cognitive ability observed in both humans and animals.
• Watermaze tasks and virtual environments are key laboratory tools for studying spatial learning.
• The hippocampus plays a crucial role in spatial learning, particularly in rapid, one-trial allocentric place learning.

Purpose of the Study:

• To review computational principles within a reinforcement learning framework for spatial navigation in watermaze tasks.
• To evaluate the efficacy and limitations of these models in reproducing behavioral and neurobiological aspects of hippocampus-dependent navigation.
• To explore how actor-critic architectures and hierarchical computations contribute to understanding flexible spatial navigation.

Main Methods:

• Review of computational models based on reinforcement learning, specifically actor-critic architectures.
• Analysis of models' ability to replicate behavioral data from watermaze and virtual delayed-matching-to-place tasks.
• Examination of neurobiological realism and mapping to hippocampal and striatal functions.

Main Results:

• Actor-critic architectures, when combined with map-like place representations, can reproduce one-trial place learning observed in watermaze and virtual tasks.
• These mechanisms align with neurobiological findings implicating the striatum and hippocampo-striatal interactions.
• Hierarchical reinforcement learning models can explain flexible navigation to familiar locations but not one-trial learning of new locations.

Conclusions:

• Actor-critic models offer a viable computational framework for understanding rapid spatial learning in navigation tasks.
• Future models for one-shot learning should integrate hippocampal plasticity mechanisms for allocentric place representation.
• Computational approaches provide valuable insights into the neural basis of spatial memory and navigation.