A Novel Functional Electrical Stimulation-Induced Cycling Controller Using Reinforcement Learning to Optimize Online...

Area of Science:

• Biomedical Engineering
• Neuroscience
• Rehabilitation Technology

Background:

• Functional Electrical Stimulation (FES) cycling is a promising rehabilitation tool for individuals with spinal cord injury (SCI).
• Traditional FES control often relies on static stimulation patterns, which may not adapt to the dynamic physiological changes during cycling.
• Optimizing stimulation parameters in real-time is crucial for improving FES cycling efficiency and user experience.

Purpose of the Study:

• To introduce a novel Reinforcement Learning (RL) controller for real-time adaptation of FES stimulation patterns during cycling.
• To investigate if a non-stationary stimulation pattern, learned by an RL agent, can better adjust electrical charge to time-varying muscle characteristics.
• To evaluate the RL controller's ability to modulate stimulation while tracking a reference pedaling cadence.

Main Methods:

• A subject with SCI (AIS-A, T8) performed overground FES-assisted cycling.
• A Proportional-Integral (PI) controller managed stimulation current amplitude, while an RL agent with a decayed-epsilon-greedy strategy explored variations in pulse amplitude and width.
• The RL agent learned to modulate electrical charge based on a predefined policy to optimize stimulation parameters.

Main Results:

• The participant successfully pedaled overground for distances exceeding 3.5 km.
• The RL agent demonstrated learning by modifying the stimulation pattern according to the predefined policy.
• The system effectively tracked the predefined pedaling cadence simultaneously with RL-driven stimulation adaptation.

Conclusions:

• The developed RL-based controller offers a simplified approach to reduce the time required for defining FES stimulation patterns.
• This method shows potential for improving FES cycling performance by enabling adaptive stimulation.
• Future research can explore more sophisticated RL algorithms and stimulation cost dynamics for enhanced efficiency.