Hybrid Bipedal Locomotion Based on Reinforcement Learning and Heuristics

Area of Science:

• Robotics
• Control Systems
• Artificial Intelligence

Background:

• Legged robot locomotion control is crucial for agile movement.
• Model-based controllers offer robustness, while reinforcement learning excels in generalization.
• Existing methods often face challenges in balancing these approaches.

Purpose of the Study:

• To propose a hybrid locomotion controller framework combining deep reinforcement learning (DRL) and heuristic policies.
• To enable adaptive training without conflicts between heuristic knowledge and learned policies.
• To improve the generalization and robustness of legged robot locomotion.

Main Methods:

• A hybrid framework integrating DRL and heuristic policies, with distinct activation phases.
• Step-by-step stochastic curriculum training in simulation.
• Domain randomization and assistive feedback loops for real-world transfer.
• Experimental validation on simulated and real Wukong-IV humanoid robots.

Main Results:

• The hybrid approach demonstrated comparable performance to end-to-end methods.
• Achieved a higher success rate and faster convergence speed in locomotion tasks.
• Reduced velocity tracking error by 60% on real humanoid robots.
• Successfully smoothed the sim-to-real transfer of locomotion policies.

Conclusions:

• The proposed hybrid controller effectively combines the strengths of DRL and heuristic methods.
• This framework facilitates robust and adaptive locomotion control for legged robots.
• The approach shows significant improvements in efficiency and accuracy for velocity tracking tasks.