A Multi-Agent Reinforcement Learning Method for Omnidirectional Walking of Bipedal Robots

Area of Science:

• Robotics
• Artificial Intelligence
• Machine Learning

Background:

• Omnidirectional walking in bipedal robots is a complex challenge.
• Existing reinforcement learning (RL) methods often rely on state machines, causing instability during gait transitions.
• Seamless omnidirectional gait and transient motion are crucial for advanced bipedal robots.

Purpose of the Study:

• To develop a novel multi-agent RL method for seamless omnidirectional walking in bipedal robots.
• To overcome the limitations of state machine-based policy switching in existing RL approaches.
• To enable smooth transitions between diverse gaits and transient motions.

Main Methods:

• Designed a multi-agent RL algorithm using an actor-critic framework with policy entropy for enhanced exploration.
• Developed a heterogeneous policy experience replay mechanism based on Euclidean distance.
• Introduced a periodic gait function and a curriculum learning method for improved policy robustness and faster convergence.

Main Results:

• The proposed method successfully achieved multiple gaits within a single policy network.
• Demonstrated smooth and seamless transitions between different bipedal robot gaits in simulation.
• Validated the effectiveness of the multi-agent RL approach in overcoming previous limitations.

Conclusions:

• The novel multi-agent RL method provides a robust solution for omnidirectional walking in bipedal robots.
• Achieved stable and smooth gait transitions, addressing a key challenge in the field.
• The approach shows significant potential for advancing bipedal robot locomotion capabilities.