From Rough to Precise: Human-Inspired Phased Target Learning Framework for Redundant Musculoskeletal Systems

Area of Science:

• Robotics
• Computational Neuroscience
• Biomechanics

Background:

• Redundant muscles in human-like robots create complex solution spaces for muscle excitation computation.
• Traditional methods like dynamic optimization and reinforcement learning face high computational costs and unstable learning in complex musculoskeletal systems.

Purpose of the Study:

• To propose a novel phased target learning framework to address challenges in computing muscle excitations for robots.
• To enhance reinforcement learning methods for more stable and efficient training of musculoskeletal robots.
• To provide insights into human motor learning mechanisms through computational modeling.

Main Methods:

• Developed a phased target learning framework guiding learners through varying difficulty levels to avoid local optima.
• Improved Q-network methods with an additional preference layer for generating continuous muscle excitations.
• Incorporated biological noise sources, inspired by the human nervous system, to improve solution space exploration.

Main Results:

• The phased target learning framework demonstrated stabilized training by preventing excitation divergence in musculoskeletal arm models.
• Enhanced exploration through biological noise sources led to reduced motion errors.
• The framework proved effective in guiding the learning process and improving computational efficiency.

Conclusions:

• The proposed phased target learning framework offers a stable and efficient approach to muscle excitation computation in robots.
• This framework shows potential for broader applications in general-purpose reinforcement learning.
• The study provides a computational interpretation of human motor learning mechanisms.