Improved double DQN with deep reinforcement learning for UAV indoor autonomous obstacle avoidance

Area of Science:

• Robotics
• Artificial Intelligence
• Computer Science

Background:

• Unmanned Aerial Vehicles (UAVs) face challenges in autonomous obstacle avoidance within complex indoor environments.
• Existing deep reinforcement learning algorithms may lack sufficient perception and learning capabilities for these scenarios.

Purpose of the Study:

• To propose an improved Double Deep Q-Network (DQN) algorithm to enhance UAV autonomous obstacle avoidance performance.
• To optimize the network model and implement a dynamic exploration strategy for improved efficiency and convergence.

Main Methods:

• Developed an improved Double DQN algorithm integrating optimized network architecture and a dynamic exploration strategy.
• Utilized AirSim and Unreal Engine 4 (UE4) to create diverse indoor simulation environments for testing.
• Evaluated performance across scenarios with varying complexity.

Main Results:

• In simpler scenarios, average cumulative reward increased by 22.88% and average safe flight distance by 23.17%.
• In complex scenarios, average cumulative reward increased by 2.66% and average safe flight distance by 2.05%.
• Significant improvements in maximum rewards and safe flight distances were observed in both scenarios.

Conclusions:

• The proposed improved Double DQN algorithm effectively enhances UAV autonomous obstacle avoidance in complex indoor settings.
• The optimized network and dynamic exploration strategy contribute to improved performance and efficiency.
• The findings demonstrate the algorithm's potential for real-world applications requiring robust indoor navigation.