A simulated environment for early development stages of reinforcement learning algorithms for closed-loop deep brain...

Area of Science:

• Neuroscience
• Biomedical Engineering
• Control Systems

Background:

• Adaptive deep brain stimulation (aDBS) shows promise for Parkinson's disease (PD) therapy, but faces challenges from non-stationary dynamics and patient heterogeneity.
• Safety constraints necessitate complex surrogate platforms for developing and validating novel aDBS control algorithms.

Purpose of the Study:

• To characterize and categorize non-stationary dynamics relevant to aDBS in Parkinson's disease.
• To develop a surrogate simulation environment for early-stage aDBS control strategy development, particularly for reinforcement learning (RL).
• To compare RL methods suitable for non-stationary dynamics in aDBS.

Main Methods:

• Characterization and categorization of non-stationary dynamics in the aDBS control problem.
• Development of a surrogate simulation environment integrating knowledge of these dynamics.
• Comparative analysis of reinforcement learning algorithms for aDBS control.

Main Results:

• The study successfully characterized and categorized key non-stationary dynamics in aDBS.
• A functional surrogate simulation environment was created to facilitate RL-based aDBS control development.
• A comparison highlighted the performance of different RL methods against identified dynamics.

Conclusions:

• Knowledge of non-stationary dynamics is crucial for effective aDBS control in Parkinson's disease.
• The developed simulation platform supports the safe and efficient development of advanced aDBS control strategies.
• Open-source code availability promotes reproducibility and adoption of the proposed methods.