Real-world humanoid locomotion with reinforcement learning.

Area of Science:

• Robotics
• Artificial Intelligence
• Machine Learning

Background:

• Classical controllers for humanoid robots struggle with generalization and adaptation to new environments.
• Autonomous humanoid robots are crucial for applications like manufacturing, elder care, and space exploration.

Purpose of the Study:

• To develop a fully learning-based approach for real-world humanoid locomotion.
• To create a controller that can adapt to diverse environments and disturbances without weight updates.

Main Methods:

• A causal transformer model was developed to predict robot actions based on historical observations and actions.
• The model was trained using large-scale model-free reinforcement learning in simulated, randomized environments.
• The controller was deployed in real-world scenarios with zero-shot adaptation.

Main Results:

• The learning-based controller successfully enabled humanoid robots to walk over various outdoor terrains.
• The controller demonstrated robustness to external disturbances.
• The model exhibited context-specific adaptation without requiring retraining.

Conclusions:

• A fully learning-based approach using causal transformers can achieve robust, adaptive humanoid locomotion in the real world.
• This method overcomes limitations of classical controllers in generalization and adaptation.
• The developed controller shows significant potential for autonomous operation in complex, unstructured environments.