Intermediate Sensory Feedback Assisted Multi-Step Neural Decoding for Reinforcement Learning Based Brain-Machine...

Area of Science:

• Neuroscience
• Machine Learning
• Biomedical Engineering

Background:

• Reinforcement-learning (RL)-based brain-machine interfaces (BMIs) show promise for clinical applications by decoding neural activity into movement intentions.
• Traditional RL-BMI tasks often use single-step rewards, unlike real-world scenarios requiring multi-step actions and internal evaluation of sensory feedback.

Purpose of the Study:

• To develop an effective tool for multi-step decoding in RL-BMIs by utilizing intermediate neural signals.
• To improve the accuracy, convergence, and stability of RL-BMI decoders for complex, multi-step tasks.

Main Methods:

• Proposed extracting intermediate guidance from the medial prefrontal cortex (mPFC) to aid multi-step decoding within an RL framework.
• Incorporated a temporal difference (TD) method into quantized attention-gated kernel reinforcement learning (QAGKRL) for credit assignment in large state-action spaces.
• Tested the approach on rat data from the primary motor cortex (M1) and mPFC during multi-step cursor control tasks.

Main Results:

• The proposed method, using intermediate mPFC signals, improved prediction accuracy by 10.9% on average compared to models using only final rewards.
• The algorithm demonstrated faster convergence and enhanced stability in decoding performance.
• Achieved an additional 18.2% increase in decoding accuracy compared to existing TD-RL methods.

Conclusions:

• Intermediate neural evaluations from brain regions like the mPFC can significantly enhance multi-step decoding in RL-BMIs.
• The developed QAGKRL with TD method offers a robust approach for credit assignment in complex BMI tasks.
• This research paves the way for more sophisticated BMI applications requiring intricate, multi-step movement control.