Acquiring musculoskeletal skills with curriculum-based reinforcement learning

Area of Science:

• Computational neuroscience
• Robotics
• Biomechanics

Background:

• Understanding biological motor control is a complex challenge.
• Musculoskeletal simulators and machine learning offer computational tools.
• Human skill learning provides a model for motor control strategies.

Purpose of the Study:

• To develop and evaluate a novel approach for controlling a realistic human hand model.
• To investigate the emergence of kinematic synergies during motor learning.
• To compare computational findings with human motor control data.

Main Methods:

• Utilized a novel curriculum learning approach to train a recurrent neural network.
• Controlled a 39-muscle realistic human hand model to perform a task (rotating Baoding balls).
• Employed selective inactivation of control signals to analyze dimensional contributions.

Main Results:

• The trained policy discovered a small number of kinematic synergies, aligning with human subject data.
• Despite no explicit bias towards low-dimensional solutions, the model exhibited synergistic control.
• Analysis revealed that more dimensions contribute to task performance than traditional synergy analysis suggests.

Conclusions:

• The study demonstrates the potential of integrating musculoskeletal simulators, reinforcement learning, and neuroscience.
• The findings advance our understanding of biological motor control and the nature of motor synergies.
• This approach offers new possibilities for studying complex motor behaviors computationally.