Reaching the limit in autonomous racing: Optimal control versus reinforcement learning.

Area of Science:

• Robotics
• Control Systems
• Machine Learning

Background:

• Designing control systems for agile mobile robots is a key challenge in robotics.
• Autonomous drone racing presents a demanding testbed for robot control systems.
• Traditional optimal control (OC) methods face limitations in complex, dynamic environments.

Purpose of the Study:

• To systematically investigate the design of control systems for agile mobile robots.
• To compare the performance of reinforcement learning (RL) and optimal control (OC) in autonomous drone racing.
• To identify the fundamental factors contributing to the success of RL over OC.

Main Methods:

• A neural network controller was trained using reinforcement learning (RL).
• The performance of the RL controller was compared against optimal control (OC) methods.
• Domain randomization was employed within the RL framework to handle model uncertainty.

Main Results:

• The RL-trained controller significantly outperformed optimal control (OC) methods in autonomous drone racing.
• RL's advantage stems from optimizing a more suitable objective, not superior optimization efficiency.
• The RL controller achieved peak acceleration exceeding 12g and a peak velocity of 108 km/h, demonstrating superhuman performance.

Conclusions:

• Reinforcement learning (RL) offers a superior approach to robot control compared to optimal control (OC) for agile systems.
• RL's ability to directly optimize task-level objectives and handle model uncertainty is crucial for advanced robotic behaviors.
• This study marks a milestone in agile robotics, highlighting RL's potential for achieving unprecedented performance and robust control.