Investigating navigation strategies in the Morris Water Maze through deep reinforcement learning

Area of Science:

• Computational neuroscience
• Artificial intelligence
• Animal behavior

Background:

• Navigation is a complex cognitive function studied extensively in both humans and animals.
• The Morris Water Maze is a standard experimental paradigm for assessing spatial learning and memory.

Purpose of the Study:

• To simulate the Morris Water Maze using deep reinforcement learning (DRL) for agent training.
• To classify and analyze navigation strategies employed by DRL agents.
• To compare agent learning dynamics and internal representations with biological data from humans and rodents.

Main Methods:

• Development of a 2D Morris Water Maze simulation environment.
• Training DRL agents using reinforcement learning algorithms.
• Implementation of automatic classification for navigation strategies.
• Design and evaluation of environment-specific auxiliary tasks for DRL agents.
• Analysis of neural network activations to identify internal representations.

Main Results:

• DRL agents demonstrated navigation strategies and learning curves mirroring those observed in human and rodent studies.
• Auxiliary tasks were developed, with some proving more beneficial and potentially biologically plausible than others.
• Internal representations within the agents' neural networks, analogous to place cells and head-direction cells, were identified.
• A correlation was found between these internal representations and the specific navigation strategies adopted by the agents.

Conclusions:

• DRL agents can effectively learn complex navigation tasks, exhibiting biologically relevant learning dynamics and strategy distributions.
• Auxiliary tasks can enhance DRL agent performance, with the most effective ones potentially reflecting biological learning mechanisms.
• The emergence of place cell- and head-direction cell-like representations in DRL agents provides insights into the neural basis of spatial navigation.