DeltaQ: Value-Guided Hebbian Learning in Spiking Neuronal Networks for Multi-Goal Navigation

Area of Science:

• Computational neuroscience
• Neural networks
• Reinforcement learning

Background:

• Animals navigate complex environments using spatial memory and internal representations.
• The hippocampal-entorhinal system is crucial for spatial navigation.
• Existing models often focus on neural dynamics, not learning in navigation tasks.

Purpose of the Study:

• To present a biologically inspired spiking neuronal network (SNN) model for navigation.
• To demonstrate how spatial representations and plasticity support learning under sparse reward conditions.
• To investigate the role of contextual modulation in supporting multiple navigation objectives.

Main Methods:

• Developed a spiking neuronal network (SNN) model integrating grid-cell-derived spatial codes, association cells, and context cells.
• Incorporated ΔQ-modulated Hebbian plasticity for learning from sparse and delayed rewards.
• Utilized a goal-conditioned Q-table to compute value changes (ΔQ) for synaptic plasticity.

Main Results:

• The model successfully generated distinct spatial representations.
• It learned efficient navigation policies in maze environments with sparse and delayed rewards.
• Contextual modulation enabled a shared network to support multiple, distinct navigation policies.

Conclusions:

• Biologically inspired spatial representations, value-guided plasticity, and contextual modulation jointly enable flexible navigation in SNNs.
• The model bridges mechanistic neural circuit models and functional reinforcement learning.
• Contextual modulation allows for task-dependent variations in navigation behavior at shared spatial locations.