Hierarchical Optimization Design for Autonomous Flight of Vision-Based Quadrotor Using Reinforcement Learning

Area of Science:

• Robotics
• Artificial Intelligence
• Control Systems

Background:

• Quadrotor drones are widely used but require improved autonomous capabilities for complex environments.
• Autonomous flight in confined spaces presents significant challenges for current drone technology.
• Monocular vision-based navigation is crucial for enabling drones to operate without external sensors.

Purpose of the Study:

• To develop and validate an intelligent control and decision-making framework for quadrotor autonomous flight.
• To enhance the safety and efficiency of quadrotor navigation in narrow, complex environments.
• To integrate advanced reinforcement learning techniques for real-time drone control and path planning.

Main Methods:

• A hierarchical reinforcement learning approach was employed, dividing the problem into control and decision layers.
• A parallel policy iteration algorithm was designed for the quadrotor's nonaffine nonlinear system, using motor speed as input.
• Autonomous decision-making was modeled as a Markov decision process, incorporating a curriculum learning mechanism and optimized proximal policy optimization (PPO).

Main Results:

• The proposed controller demonstrated online learning capabilities, improving fundamental control performance.
• The curriculum learning mechanism effectively addressed sparse reward challenges in the decision-making process.
• Optimized PPO enhanced the efficiency of developing autonomous flight capabilities for the quadrotor.

Conclusions:

• The developed intelligent control and decision methods significantly improve quadrotor autonomous flight in complex environments.
• The hierarchical reinforcement learning framework provides a robust solution for narrow space traversal tasks.
• Simulation results validate the effectiveness and potential of the proposed approach for real-world applications.