LORM: a novel reinforcement learning framework for biped gait control.

Area of Science:

• Robotics
• Artificial Intelligence
• Control Systems

Background:

• Legged robots offer superior adaptability to varied terrains compared to wheeled robots.
• Traditional motion controllers face challenges due to complex robot dynamics.
• Reinforcement learning (RL) presents a promising approach to simplify dynamics design and enhance controller autonomy and robustness.

Purpose of the Study:

• To propose a novel RL-based framework, Learn and Outperform Reference Motion (LORM), for biped robot gait control.
• To leverage prior knowledge of reference motion to improve RL training efficiency and performance.
• To enhance the robustness and adaptability of legged robots in complex environments.

Main Methods:

• Developed LORM, an RL framework integrating reference motion knowledge for gait control.
• Optimized the RL environment through state-action space pruning, reward shaping, and episode design.
• Implemented performance enhancements: random state initialization (RSI), joint angle noise, and gait symmetrization.

Main Results:

• On the Darwin-op robot, LORM achieved 0.488 m/s walking velocity, a 5.8x improvement over traditional controllers.
• Directional accuracy improved by 87.3%, with velocity performance exceeding rated maximums and prior works.
• The method demonstrated over 95% tracking accuracy for specific velocities and maintained stability across diverse terrains and external forces.

Conclusions:

• LORM significantly advances RL-based gait control for biped robots, achieving state-of-the-art performance.
• The framework's enhancements effectively address RL training challenges, improving efficiency and robustness.
• The proposed method demonstrates exceptional adaptability and stability, paving the way for more capable legged robots.