Inertia-Constrained Reinforcement Learning to Enhance Human Motor Control Modeling

Area of Science:

• Biomechanics
• Robotics
• Artificial Intelligence

Background:

• Locomotor impairment is a major cause of disability, impacting quality of life.
• Simulating human locomotion is crucial for studying musculoskeletal drivers and clinical conditions.
• Current reinforcement learning (RL) simulations lack natural human movement due to insufficient reference data.

Purpose of the Study:

• To improve the realism and efficiency of simulated human locomotion.
• To develop an RL strategy that incorporates reference motion data for more natural movement simulation.
• To accelerate the training process for locomotion simulation models.

Main Methods:

• Designed a novel reward function combining trajectory optimization rewards (TOR) and bio-inspired rewards.
• Integrated reference motion data captured by a single Inertial Measurement Unit (IMU) sensor on the pelvis.
• Adapted the reward function using prior research on walking simulations for TOR.

Main Results:

• Simulated agents with the modified reward function demonstrated superior mimicry of participant IMU data, indicating more realistic locomotion.
• The inclusion of IMU data as a bio-inspired cost accelerated agent convergence during training.
• Models trained with reference motion data exhibited faster convergence compared to those without.

Conclusions:

• The proposed reward function significantly enhances the realism of simulated human locomotion.
• Incorporating reference motion data improves the efficiency and speed of RL-based locomotion simulations.
• This approach facilitates quicker and more effective simulation of human movement across diverse environments.