Grid cells, place cells, and geodesic generalization for spatial reinforcement learning.

Area of Science:

• Neuroscience
• Computational Neuroscience
• Cognitive Science

Background:

• Reinforcement learning (RL) models brain mechanisms for advantageous choice learning.
• Efficient task representation is crucial for complex RL.
• Hippocampal place cells and entorhinal grid cells are key spatial representations.

Purpose of the Study:

• To investigate how hippocampal place cells and entorhinal grid cells support RL in spatial navigation.
• To explore the role of spatial representations in enabling efficient generalization for RL.
• To determine if Euclidean or geodesic representations are more beneficial for RL.

Main Methods:

• Simulations of navigational tasks using RL.
• Analysis of place cell and grid cell representations as basis functions for value approximation.
• Comparison of Euclidean and geodesic distance metrics in RL simulations.

Main Results:

• Diffuse tuning of individual neurons enhances RL efficiency by promoting generalization.
• Geodesic representations, unlike Euclidean ones, improve RL by respecting environmental layout and obstacles.
• Simulations confirm that Euclidean representations can hinder RL by generalizing inappropriately across barriers.

Conclusions:

• Spatial representations in the brain are adapted for RL, with diffuse tuning aiding generalization.
• Geodesic distance is a more relevant metric than Euclidean distance for value approximation in RL for navigation.
• The findings suggest that place and grid cell responses should be modulated by geodesic distances, offering new predictions for spatial coding.