Enhancement of Hippocampal Spatial Decoding Using a Dynamic Q-Learning Method With a Relative Reward Using Theta...

Area of Science:

• Computational Neuroscience
• Machine Learning
• Neuroscience

Background:

• Hippocampal place cells and interneurons encode spatial information through stable place fields and theta phase precession.
• These neurons, particularly in CA1, represent an animal's location and prospective goal information.
• Reinforcement learning (RL), specifically Q-learning, is utilized for navigation modeling.

Purpose of the Study:

• To address limitations of traditional Q-learning (τQ-learning) in location accuracy and convergence rates.
• To introduce and evaluate a revised dynamical Q-learning (δQ-learning) algorithm for adaptive reward function assignment.
• To enhance trajectory reconstruction and prediction performance in spatial navigation tasks.

Main Methods:

• Developed δQ-learning, adapting the reward function dynamically for improved decoding.
• Utilized firing rate as input for the δQ-learning neural network to predict movement direction.
• Employed theta phase precession as input for the reward function to update δQ-learning weights.
• Compared trajectory prediction accuracy using root mean squared error (RMSE) between δQ- and τQ-learning.

Main Results:

• δQ-learning demonstrated significantly higher prediction accuracy and faster convergence rates than τQ-learning across all cell types.
• Integrating place cells and interneurons with theta phase precession further improved convergence and accuracy.
• The δQ-learning algorithm proved to be a rapid and precise method for trajectory reconstruction and prediction.

Conclusions:

• The proposed δQ-learning algorithm offers a superior approach to traditional Q-learning for spatial navigation modeling.
• Adaptive reward assignment and integration of neural data (firing rate, phase precession) are key to enhanced performance.
• This method holds promise for accurate trajectory reconstruction and prediction in neuroscience research.