A confidence metric for using neurobiological feedback in actor-critic reinforcement learning based brain-machine...

Area of Science:

• Neuroscience
• Biomedical Engineering
• Machine Learning

Background:

• Brain-Machine Interfaces (BMIs) offer functional restoration for individuals with paralysis.
• Current BMIs face challenges in daily living (ADL) due to extensive calibration and external training input requirements.
• Reinforcement Learning (RL) offers adaptive training for BMIs but is sensitive to feedback accuracy.

Purpose of the Study:

• To develop an adaptive BMI that overcomes limitations of critic feedback inaccuracies in RL-based systems.
• To enhance the accuracy and autonomy of BMIs for seamless integration into daily activities.
• To reduce the need for external signals and extensive calibration in BMI technology.

Main Methods:

• Developed an adaptive actor-critic BMI incorporating a critic confidence measure.
• The confidence measure assesses the appropriateness of feedback for updating actor decoding parameters.
• Validated the system using synthetic neural data (Izhikevich model, Gaussian noise) and non-human primate reaching task data.

Main Results:

• The adaptive BMI with critic confidence consistently outperformed systems without it across all tested datasets.
• Critic accuracy was no longer the limiting factor for overall BMI performance.
• The developed confidence measure effectively handled inaccuracies in critic feedback.

Conclusions:

• The proposed technique enables more accurate RL-based BMIs by managing critic feedback uncertainty.
• This approach significantly reduces the need for external training signals and complex calibration procedures.
• Suggests potential for developing autonomous BMIs suitable for real-world applications and daily living.