Feedback for reinforcement learning based brain-machine interfaces using confidence metrics.

Area of Science:

• Neuroscience
• Biomedical Engineering
• Machine Learning

Background:

• Brain-machine interfaces (BMIs) require high autonomy for daily living applications in paralyzed individuals.
• Extracting and utilizing feedback is a key challenge in developing autonomous BMIs.
• Current BMIs often rely on error signals, limiting adaptive capabilities.

Purpose of the Study:

• To develop an autonomous BMI system that utilizes evaluative brain feedback to adapt its decoding algorithm.
• To incorporate feedback quantity and quality into a reinforcement learning (RL) controller architecture.
• To enhance BMI performance and reduce reliance on extensive initial training.

Main Methods:

• Utilized an Actor-Critic RL architecture inspired by the brain's perception-action-reward cycle.
• Incorporated a confidence metric to manage the accuracy and reliability of biological reward signals from the nucleus accumbens (NAcc).
• Employed biologically realistic models (Izhikevich, Humphries) for synthetic data generation (motor imagery, NAcc signals).

Main Results:

• Demonstrated improved BMI performance by using a threshold to reject erroneous feedback.
• Showcased enhanced system stability through the judicious use of feedback with a threshold.
• Significantly decreased initial decoder training time to approximately 10 trials per session.

Conclusions:

• The proposed method represents a significant step towards autonomous BMIs.
• The approach reduces the extensive training typically required for BMI systems.
• The thresholding method is applicable to any decoder with imperfect feedback, improving stability and avoiding errors.