Simulating human walking: a model-based reinforcement learning approach with musculoskeletal modeling.

Area of Science:

• Biomechanics
• Robotics
• Computational Modeling

Background:

• Reinforcement learning (RL) advances control models for human movement simulation.
• Biomechanical models are crucial for realistic human-like motion generation.
• Integrating RL with musculoskeletal models enhances gait simulation accuracy.

Purpose of the Study:

• To develop control modes for human walking using an integrated RL and musculoskeletal model.
• To investigate the model's ability to generate naturalistic gait without reference motion data.
• To explore the potential for modeling pathological gait.

Main Methods:

• A musculoskeletal model (trunk, pelvis, legs) was combined with an RL algorithm.
• Simulations were performed without target speed, then with imposed speeds.
• The Markov decision process problem was solved using covariance matrix adaptation evolution strategy.

Main Results:

• The model self-selected a stable walking speed of 1.45 m/s.
• Simulated hip and knee kinematics closely matched experimental data.
• Ankle kinematics prediction showed less accuracy compared to hip and knee.

Conclusions:

• The integrated RL and musculoskeletal model successfully simulated human walking.
• The framework demonstrates potential for predicting gait abnormalities, such as those from muscle weakness.
• This approach advances the development of sophisticated human movement models.