Learning to Ascend Stairs and Ramps: Deep Reinforcement Learning for a Physics-Based Human Musculoskeletal Model

Area of Science:

• Biomechanics
• Robotics
• Machine Learning

Background:

• Human locomotion involves complex musculoskeletal dynamics.
• Simulating and replicating human movement is crucial for prosthetics and robotics.
• Physics-based models offer realistic biomechanical representations.

Purpose of the Study:

• To apply deep reinforcement learning (DRL) for human musculoskeletal model locomotion.
• To train a model for stair and ramp ascent using DRL and imitation learning.
• To validate the model's performance against experimental data.

Main Methods:

• Developed a physics-based human musculoskeletal model in OpenSim.
• Utilized proximal policy optimization (PPO) with imitation learning for training.
• Incorporated elastic foundation contact dynamics for realistic interactions.
• Trained the model using a public dataset of human locomotion.

Main Results:

• The DRL model learned to ascend stairs and ramps effectively.
• Achieved muscle forces comparable to healthy human subjects.
• Demonstrated forward dynamics similar to experimental data, with correlations of 0.82 (stairs) and 0.58 (ramps).

Conclusions:

• DRL is a viable method for training physics-based musculoskeletal models for locomotion.
• The trained model shows potential for simulating human-like movement on varied terrains.
• This approach advances biomechanical simulation and robotic control applications.