Training an Actor-Critic Reinforcement Learning Controller for Arm Movement Using Human-Generated Rewards.

Area of Science:

• Neuroscience
• Biomedical Engineering
• Rehabilitation Technology

Background:

• Functional Electrical Stimulation (FES) uses neuroprostheses to restore movement in individuals with spinal cord injury.
• Current FES controllers struggle to adapt to individual physiological variations and user preferences.
• Reinforcement learning (RL) offers a promising approach by incorporating human feedback to shape controller behavior.

Purpose of the Study:

• To investigate the efficacy of using human-assigned rewards to train RL-based FES controllers.
• To compare the learning performance of RL controllers trained with human rewards versus algorithm-generated rewards.
• To evaluate the impact of reward characteristics on controller learning success.

Main Methods:

• Ten neurologically intact participants provided subjective numerical rewards to train RL controllers.
• A planar musculoskeletal human arm simulation was used to evaluate goal-oriented reaching tasks.
• Controller performance was assessed based on target success, time to reach, and target overshoot.
• Comparison was made between human-trained RL controllers and algorithm-trained RL controllers.

Main Results:

• Both human-trained and algorithm-trained RL controllers demonstrated efficient learning.
• Both RL controller groups significantly outperformed standard controllers in reaching tasks.
• No significant difference in learning efficiency was observed between human-trained and algorithm-trained controllers.
• Reward positivity and consistency did not correlate with learning success.

Conclusions:

• Human rewards are effective for training RL-based FES controllers.
• RL controllers trained with human feedback can restore voluntary movement in simulated paralyzed limbs.
• This approach holds potential for personalized FES control strategies in spinal cord injury rehabilitation.