Navigation in Unknown Dynamic Environments Based on Deep Reinforcement Learning.

Area of Science:

• Robotics
• Artificial Intelligence
• Machine Learning

Background:

• Robots often struggle with navigation in complex, dynamic environments due to incomplete sensor data and moving obstacles.
• Existing algorithms may fail to converge or avoid local minima in environments with sparse rewards or noisy estimations.

Purpose of the Study:

• To develop a novel Deep Reinforcement Learning algorithm for robust robot navigation in unknown dynamic environments.
• To enhance a robot's temporal reasoning and environmental modeling capabilities.
• To address challenges of non-convergence policies and sparse rewards in robot navigation.

Main Methods:

• Proposes MK-A3C (Memory and Knowledge-based Asynchronous Advantage Actor-Critic), a novel Deep Reinforcement Learning algorithm.
• Integrates a GRU-based memory neural network for enhanced temporal reasoning and environmental modeling.
• Combines a domain knowledge-based reward function and transfer learning for improved training stability.

Main Results:

• MK-A3C successfully navigates non-holonomic robots with continuous control in unknown dynamic environments with moving obstacles.
• The algorithm demonstrates improved rationality and avoids local minima traps through enhanced memory and environmental estimation.
• Simulation experiments show superior performance compared to existing methods in challenging navigation tasks.

Conclusions:

• MK-A3C provides an efficient and robust solution for robotic navigation in complex, dynamic environments.
• The integration of memory and knowledge-based components significantly improves navigation performance and stability.
• The algorithm effectively handles kinetic constraints and moving objects while outputting continuous acceleration commands.