A parallel heterogeneous policy deep reinforcement learning algorithm for bipedal walking motion design

Area of Science:

• Robotics
• Artificial Intelligence
• Control Systems

Background:

• Biped robot locomotion presents significant challenges due to complex dynamics and non-linear characteristics.
• Existing gait optimization methods struggle with adaptability across diverse environments and robot models.

Purpose of the Study:

• To develop an efficient and robust gait optimization algorithm for biped robots.
• To improve walking speed, stability, and adaptability of biped robots in varied conditions.

Main Methods:

• A parallel heterogeneous policy Deep Reinforcement Learning (DRL) algorithm utilizing the Deep Deterministic Policy Gradient (DDPG) architecture.
• Implementation of shared networks for enhanced training efficiency and heterogeneous experience replay for optimized experience utilization.
• Design of a periodic gait based on sinusoidal curves, incorporating foot lift height, walking period, speed, and ground contact force.

Main Results:

• The proposed DRL algorithm successfully optimized biped robot gaits, leading to faster locomotion.
• Experimental results demonstrate significantly improved walking stability across different environments and robot models.
• The unified gait optimization framework proved effective in diverse simulation scenarios on the RoboCup3D platform.

Conclusions:

• The developed parallel heterogeneous DRL approach offers a powerful solution for biped robot gait optimization.
• The method enhances key performance metrics, making biped robots more practical for real-world applications.
• The unified framework provides a versatile platform for future research in bipedal locomotion.