Muscle Synergy-Guided Reinforcement Learning for Embodied Musculoskeletal Motion Skill Learning

Area of Science:

• Robotics
• Biomechanics
• Machine Learning

Background:

• Acquiring human-like motor skills in complex musculoskeletal models is hindered by high dimensionality and actuator redundancy.
• Existing methods struggle with the intricate coordination required for naturalistic movement.

Purpose of the Study:

• To develop a reinforcement learning framework that integrates neurophysiological priors, specifically muscle synergies, to improve motor skill learning in embodied models.
• To enhance the efficiency, accuracy, and interpretability of motor control in simulated musculoskeletal systems.

Main Methods:

• A synergy-guided reinforcement learning framework was developed, incorporating muscle synergies as physiological priors into the control policy.
• The framework uses coordinated muscle activation patterns to guide learning, with separate synergy-guided and residual control components.
• Four badminton strokes (forehand/backhand, inward/outward net slices) and a forehand high serve were used as benchmark tasks.

Main Results:

• The proposed method achieved high accuracy, with an average root mean square error below 0.015 radians across all tasks.
• It outperformed the proximal policy optimization (PPO) baseline in trajectory accuracy, energy efficiency (up to 14.9% reduction), and convergence speed.
• Learned muscle synergies showed moderate resemblance to human synergies, suggesting biological plausibility and interpretability.

Conclusions:

• Integrating neurophysiological priors into reinforcement learning offers a viable path for efficient, interpretable, and human-like motor control.
• This approach has significant potential for applications in motor skill assessment, human-machine interfaces, and rehabilitation technologies.