Creating a Reinforcement Learning Controller for Functional Electrical Stimulation of a Human Arm.

Area of Science:

• Rehabilitation Engineering
• Computational Neuroscience
• Robotics

Background:

• Human arm dynamics in Functional Electrical Stimulation (FES) trials exhibit significant variability.
• Developing adaptive controllers is crucial for consistent and effective FES-assisted movement.
• Existing control methods may struggle to compensate for real-time changes in biological systems.

Purpose of the Study:

• To investigate the efficacy of Reinforcement Learning (RL) for creating adaptive controllers for FES-controlled human arms.
• To develop an RL-based controller capable of adapting to significant and clinically relevant changes in arm dynamics.
• To evaluate the controller's performance in simulation under varying dynamic conditions.

Main Methods:

• A two-dimensional arm model with Hill-based muscle dynamics was utilized for simulations.
• An actor-critic architecture employing artificial neural networks was implemented for the RL controller.
• The RL agent was initially trained with a Proportional Derivative (PD) controller as a supervisor.
• Adaptation capabilities were tested by introducing dynamic changes and evaluating performance without supervision.

Main Results:

• The actor-critic RL controller demonstrated the ability to adapt to altered arm dynamics in simulation.
• Supervised pre-training facilitated initial controller learning.
• Unsupervised adaptation to new dynamics was achieved within a reasonable number of simulation episodes.
• The controller showed robustness to clinically relevant changes in arm biomechanics.

Conclusions:

• Reinforcement Learning offers a promising approach for developing adaptive controllers in FES applications.
• The developed RL controller can effectively compensate for changing human arm dynamics.
• This adaptive control strategy has the potential to enhance the performance and reliability of FES systems for rehabilitation and assistance.