Deep Reinforcement Learning Approach with Multiple Experience Pools for UAV's Autonomous Motion Planning in Complex...

Area of Science:

• Robotics
• Artificial Intelligence
• Aerospace Engineering

Background:

• Autonomous motion planning (AMP) is crucial for unmanned aerial vehicles (UAVs) to navigate complex environments without human intervention.
• Deep reinforcement learning (DRL) has shown promise for AMP but often requires extensive training data and time.
• Existing DRL methods face challenges in simplified environments and struggle with real-world complexities.

Purpose of the Study:

• To develop a novel DRL framework that accelerates learning for UAV autonomous motion planning.
• To leverage human expert experiences to improve the efficiency and effectiveness of DRL algorithms.
• To enable UAVs to perform complex tasks in unknown environments through enhanced DRL.

Main Methods:

• Proposed a Multiple Experience Pools (MEP) framework integrated with DRL.
• Developed a MEP-DRL algorithm based on the Deep Deterministic Policy Gradient (DDPG) algorithm.
• Utilized model predictive control and simulated annealing to generate expert experiences for training.

Main Results:

• The MEP-DRL algorithm demonstrated a performance improvement exceeding 20% compared to the state-of-the-art DDPG.
• Trained UAVs successfully completed diverse tasks in complex, unknown simulation environments.
• The framework significantly speeds up the learning process by incorporating expert experiences.

Conclusions:

• The MEP-DRL framework offers a viable solution for enhancing UAV autonomous motion planning.
• Leveraging expert experiences is effective in accelerating DRL training for complex navigation tasks.
• The proposed method enables robust and stable UAV operation in challenging, real-world-like scenarios.