Generating Human Arm Kinematics Using Reinforcement Learning to Train Active Muscle Behavior in Automotive Research

Area of Science:

• Biomechanics
• Computational modeling
• Robotics

Background:

• Computational human body models (HBMs) are crucial for predicting occupant responses in automotive crash simulations.
• Incorporating active muscle control into HBMs enhances biofidelic kinematic predictions during vehicle maneuvers.
• Traditional controllers have limitations in accurately mimicking active muscle behavior across diverse loading conditions.

Purpose of the Study:

• To develop an active muscle controller using reinforcement learning (RL) for human body models.
• To demonstrate the efficacy of the RL muscle activation control (RL-MAC) approach in generating accurate human kinematics.
• To evaluate the controller's robustness under various loading conditions, including simulated automotive impacts.

Main Methods:

• Developed an active muscle controller based on the deep deterministic policy gradient (DDPG) reinforcement learning algorithm.
• Utilized a multibody human arm model trained to perform goal-directed elbow rotation.
• Investigated muscle activation using independent muscles and antagonistic muscle groups recruitment schemes.

Main Results:

• The RL-MAC controller successfully generated accurate human kinematics, enabling the arm model to reach target positions with or without external loads.
• Simulations demonstrated the controller's ability to maintain desired joint angles under constant external loads.
• The trained RL-MAC showed robustness in a simplified automotive impact scenario, maintaining elbow joint angle control.

Conclusions:

• The proposed RL-MAC approach offers a robust method for active muscle control in HBMs.
• This reinforcement learning strategy enhances the biofidelity of human body models for automotive safety research.
• The RL-MAC approach shows promise for improving predictions of human biomechanical responses in crash environments.