A Hierarchical Framework for Quadruped Robots Gait Planning Based on DDPG.

Area of Science:

• Robotics
• Artificial Intelligence
• Control Systems

Background:

• Legged robot control is advancing with reinforcement learning (RL).
• Quadruped robots present control challenges due to continuous states and large action spaces.
• Existing RL methods struggle with optimal control for quadrupedal locomotion.

Purpose of the Study:

• To introduce a hierarchical reinforcement learning framework for optimal motion control in quadruped robots.
• To address the complexities of continuous states and vast action spaces in quadruped control.
• To improve the motion performance of quadruped robots using advanced RL techniques.

Main Methods:

• Developed a hierarchical reinforcement learning framework integrating Deep Deterministic Policy Gradient (DDPG).
• Incorporated a high-level planner for ideal motion parameter generation.
• Utilized a low-level controller with Model Predictive Control (MPC) and PD controllers for precise force and torque calculation via inverse kinematics.

Main Results:

• The hierarchical framework demonstrated superior motion performance in simulations.
• Performance was significantly better than using the Deep Deterministic Policy Gradient (DDPG) method alone.
• The system effectively generated joint motor torques through inverse kinematics for locomotion.

Conclusions:

• The proposed hierarchical reinforcement learning framework offers enhanced control for quadruped robots.
• This approach successfully overcomes limitations of simpler RL controllers in complex robotic systems.
• The integration of MPC and DDPG provides a robust solution for optimal quadrupedal motion control.