Lessons from reinforcement learning for biological representations of space

Area of Science:

• Neuroscience
• Artificial Intelligence
• Cognitive Science

Background:

• Neuroscience traditionally models spatial representation using various coordinate frames.
• Reinforcement learning (RL) offers a novel approach to spatial perception and navigation.
• Current RL models often avoid building explicit 3D 'maps'.

Purpose of the Study:

• To investigate RL methods that use image-based rewards for spatial tasks.
• To evaluate the geometric consistency of learned spatial representations.
• To explore alternatives to the 'cognitive map' theory in neuroscience.

Main Methods:

• Focus on RL agents rewarded for reaching target images.
• Testing geometric consistency through interpolation of learned locations.
• Introducing a hand-crafted, geometrically consistent representation.
• Analyzing the impact of feature persistence information on performance.

Main Results:

• RL methods without 3D maps can support geometrically consistent spatial tasks.
• A designed, geometrically consistent representation improved performance.
• Information about feature persistence (e.g., distant features) enhanced geometric task performance.
• Demonstrated effective non-Cartesian spatial representations.

Conclusions:

• Reinforcement learning provides a viable framework for studying spatial representations.
• Non-Cartesian, learned representations are promising alternatives to cognitive maps.
• These findings stimulate research into new models for spatial perception and navigation.