Optimizing microstimulation using a reinforcement learning framework.

Area of Science:

• Neuroscience
• Biomedical Engineering
• Rehabilitation Engineering

Background:

• Neuroprosthetics aim to restore lost function, but lack sensory feedback for closed-loop control.
• Current systems often rely on feedforward control, limiting natural motor system interaction.
• Restoring somatosensory feedback is crucial for enhancing neuroprosthetic functionality and user experience.

Purpose of the Study:

• To develop an online method for selecting optimal electrical microstimulation parameters for sensory feedback in neuroprosthetics.
• To utilize reinforcement learning to efficiently explore the parameter space and identify effective stimulation settings.
• To avoid the need for explicit neural response models during parameter optimization.

Main Methods:

• Proposed reinforcement learning as a framework to balance exploration and exploitation of microstimulation parameters.
• Treated the parameter selection task as a k-armed bandit problem.
• Utilized offline data from natural touch and thalamic microstimulation for preliminary analysis.

Main Results:

• The reinforcement learning algorithm consistently selected the best-matching stimulation parameters.
• The algorithm identified optimal parameters from 68 different forms after 334 realizations.
• Demonstrated efficiency in exploring the parameter space and concentrating on promising parameter forms.

Conclusions:

• Reinforcement learning provides an effective framework for online optimization of neuroprosthetic sensory feedback.
• This approach enables efficient discovery of effective microstimulation parameters without explicit neural models.
• The findings support the potential for enhanced closed-loop control in neuroprosthetics through data-driven sensory feedback restoration.