Sample-Efficient Reinforcement Learning Controller for Deep Brain Stimulation in Parkinson's Disease.

Area of Science:

• Neuroscience
• Biomedical Engineering
• Computational Neuroscience

Background:

• Deep brain stimulation (DBS) is a standard Parkinson's disease (PD) treatment, but current systems are not adaptive or personalized.
• Adaptive DBS (aDBS) uses biomarkers like beta-band oscillations for dynamic stimulation control.
• Reinforcement learning (RL) offers potential for personalized aDBS but faces challenges in sample efficiency and hardware deployment.

Purpose of the Study:

• To introduce SEA-DBS, a novel actor-critic framework designed to overcome limitations in RL-based adaptive neurostimulation.
• To enhance sample efficiency, exploration robustness, and hardware compatibility for resource-constrained neuromodulation.

Main Methods:

• Developed a sample-efficient actor-critic framework (SEA-DBS) integrating a predictive reward model.
• Implemented Gumbel-Softmax-based exploration for stable, differentiable policy updates in binary action spaces.
• Evaluated SEA-DBS using a biologically realistic simulation of Parkinsonian basal ganglia activity.

Main Results:

• SEA-DBS demonstrated faster convergence compared to existing RL methods.
• Achieved stronger suppression of pathological beta-band oscillations in simulations.
• Showcased resilience to post-training FP16 quantization, indicating hardware compatibility.

Conclusions:

• SEA-DBS presents a practical and effective RL-based framework for real-time, resource-constrained adaptive deep brain stimulation.
• The proposed methods address key challenges in applying RL to adaptive neurostimulation for Parkinson's disease.