Optimum trajectory learning in musculoskeletal systems with model predictive control and deep reinforcement learning.

Area of Science:

• Robotics
• Computational Neuroscience
• Biomechanics

Background:

• Musculoskeletal control involves managing complex, high-degree-of-freedom systems with redundant muscles.
• Controlling skeletal joints with antagonistic muscles presents an ill-posed, nonlinear challenge.
• Existing methods struggle with the redundancy inherent in muscle control.

Purpose of the Study:

• To develop a robust computational framework for musculoskeletal control addressing muscle redundancy.
• To create energy-efficient skeletal trajectories mimicking human movement.
• To map these trajectories to muscle activation sequences using deep reinforcement learning.

Main Methods:

• Implemented a twofold framework: Model Predictive Control (MPC) for trajectory optimization and Deep Reinforcement Learning (DRL) for muscle control.
• Utilized MPC to generate energy-efficient skeletal trajectories.
• Employed DRL to derive muscle activation sequences, integrating state and control inputs in a closed-loop system.

Main Results:

• The framework successfully generated diverse, optimal, and human-like skeletal movement trajectories.
• Simulation results confirmed the framework's capability across various dynamic movements, including human arm movement and obstacle avoidance.
• Closed-loop integration of state and control inputs proved crucial for efficient muscle stimulus construction.

Conclusions:

• The proposed framework offers a computational pipeline to study musculoskeletal control, complementing experimental motion capture data.
• It enables in-silico experiments for human movement analysis and the study of muscle activation ranges.
• The approach has potential applications in upper-arm rehabilitation with assistive robots and digital twin development.