Reinforcement Learning based Decoding Using Internal Reward for Time Delayed Task in Brain Machine Interfaces.

Area of Science:

• Neuroscience
• Machine Learning
• Biomedical Engineering

Background:

• Reinforcement learning (RL) in brain-machine interfaces (BMIs) typically requires immediate reward delivery.
• Existing RL algorithms are inefficient for tasks with delayed rewards.
• The medial prefrontal cortex (mPFC) is implicated in assigning credit to intermediate steps for delayed reward tasks.

Purpose of the Study:

• To simulate medial prefrontal cortex (mPFC) activity as intermediate rewards for training RL-based decoders.
• To improve RL decoder efficiency in delayed reward tasks within BMIs.
• To enhance the interpretation of neural signals into movement intentions.

Main Methods:

• Utilized a support vector machine (SVM) to identify reward expectation from mPFC activity.
• Employed Sarsa-style attention-gated reinforcement learning (SAGREL) as the RL decoder.
• Applied the method to in vivo primary motor cortex (M1) and mPFC data from rats performing a two-step movement task.

Main Results:

• SAGREL utilizing mPFC-derived intermediate rewards achieved 66.8% ± 2.0% prediction accuracy.
• This significantly outperformed RL trained solely on end-of-trial rewards (45.9% ± 1.2%).
• Demonstrated the efficacy of intermediate rewards in enhancing RL performance for delayed tasks.

Conclusions:

• Simulating mPFC activity as intermediate rewards is a viable strategy for delayed reward RL in BMIs.
• This approach enhances the accuracy of neural signal interpretation for movement intention.
• Highlights the potential of incorporating prefrontal cortex functionality into BMI algorithms.