Learning agile and dynamic motor skills for legged robots

Area of Science:

• Robotics
• Artificial Intelligence
• Control Theory

Background:

• Legged robots present significant control challenges, with human-engineered methods struggling to replicate animal agility.
• Reinforcement learning (RL) offers a promising alternative for developing sophisticated control policies with less manual design.
• Current RL applications for legged robots are largely confined to simulations due to the complexity and cost of real-world training.

Purpose of the Study:

• To introduce a method for training neural network policies in simulation and transferring them to real-world legged robots.
• To overcome the limitations of training RL policies directly on expensive and complex physical systems.
• To enable faster, automated, and cost-effective data generation for policy training.

Main Methods:

• Training a neural network control policy within a simulated environment.
• Transferring the trained policy from simulation to a state-of-the-art quadrupedal robot (ANYmal).
• Utilizing fast, automated data generation schemes for efficient policy optimization.

Main Results:

• The quadrupedal robot ANYmal demonstrated advanced locomotion skills beyond previous benchmarks.
• ANYmal precisely and energy-efficiently followed high-level body velocity commands.
• The robot achieved faster running speeds and improved recovery from falls in complex scenarios.

Conclusions:

• Simulation-to-real transfer of RL policies is a viable and effective approach for advancing legged robot capabilities.
• The proposed method significantly enhances the performance and versatility of legged robots.
• This work paves the way for more agile, robust, and efficient legged robotic systems.