Kernel Temporal Difference based Reinforcement Learning for Brain Machine Interfaces.

Area of Science:

• Neuroscience
• Machine Learning
• Biomedical Engineering

Background:

• Brain-machine interfaces (BMIs) allow individuals with disabilities to control devices using neural signals.
• Reinforcement learning (RL) offers a promising approach for BMIs, enabling learning without physical limb movement.
• Existing RL decoders struggle with tasks requiring delayed reward feedback, leading to prolonged training and local minima issues.

Purpose of the Study:

• To address the temporal credit assignment problem in RL-based BMIs for tasks with delayed rewards.
• To enhance the learning efficiency and performance of BMIs in multi-step tasks.

Main Methods:

• Integration of the temporal difference (TD) method into Quantized Attention-Gated Kernel Reinforcement Learning (QAGKRL).
• Utilization of a kernel network for global linear structure and a softmax policy for efficient state-action exploration via TD error.
• Simulation of a center-out task with delayed rewards to evaluate the algorithm.

Main Results:

• The proposed TD-enhanced QAGKRL algorithm achieved a prediction accuracy of 96.2% ± 0.77% on simulated data.
• The novel method significantly outperformed two state-of-the-art models in the simulated task.
• Demonstrated effectiveness in solving the temporal credit assignment problem for delayed reward scenarios.

Conclusions:

• The developed kernel temporal difference RL method offers a significant advancement for BMIs.
• This approach holds potential for enabling more effective online continuous decoding in BMIs for multi-step tasks with delayed rewards.
• Improves learning capabilities for individuals with motor disabilities using BMI technology.