Robot-assisted motor training: assistance decreases exploration during reinforcement learning

Area of Science:

• Robotics
• Neurorehabilitation
• Motor Learning

Background:

• Reinforcement learning (RL) is a key component of motor learning.
• Robotic therapy devices offer potential for neurorehabilitation by manipulating motor learning.
• Current RL systems may not adequately adapt to individual trainee abilities.

Purpose of the Study:

• To investigate the efficacy of RL in robotic therapy for neurorehabilitation.
• To develop and evaluate a novel reward equalization algorithm for RL-based motor learning.
• To assess the impact of reward assistance and anti-slacking mechanisms on learning and exploration.

Main Methods:

• Developed an RL system for trainees to learn target movements.
• Implemented a novel reward algorithm adjusting rewards based on individual performance relative to personal best.
• Conducted an experiment with 21 unimpaired subjects to test the system and its components.

Main Results:

• All subjects learned the target movement, with or without reward equalization.
• Artificially increasing rewards reduced exploration and slowed learning, especially with changing targets.
• An anti-slacking algorithm further impeded learning.

Conclusions:

• While RL shows promise in robotic neurorehabilitation, excessive reward assistance can impede learning.
• Reward equalization based on relative performance did not hinder learning in this study.
• Mechanisms designed to facilitate reward acquisition or prevent slacking may inadvertently reduce crucial exploration in RL.