Deep Deterministic Policy Gradient-Based Autonomous Driving for Mobile Robots in Sparse Reward Environments.

Area of Science:

• Robotics
• Artificial Intelligence
• Machine Learning

Background:

• Autonomous mobile robots often face performance degradation due to sparse reward problems in path planning.
• Deep deterministic policy gradient (DDPG) is a reinforcement learning algorithm suitable for continuous control tasks.
• Hindsight experience replay (HER) is a technique designed to improve learning from sparse rewards.

Purpose of the Study:

• To propose a DDPG-based path-planning method for mobile robots incorporating HER.
• To address and mitigate the performance issues caused by sparse rewards in autonomous driving scenarios.
• To evaluate the effectiveness of the proposed method on a TurtleBot3 robot in a simulated environment.

Main Methods:

• A DDPG algorithm with an actor-critic architecture was employed, utilizing a fully connected neural network.
• The hindsight experience replay (HER) technique was integrated to generate additional learning experiences from failed attempts.
• Laser sensor data was used for environment recognition, enabling the robot to navigate towards its destination.
• Experiments were conducted using a TurtleBot3 robot within a Gazebo simulation environment.

Main Results:

• The HER technique significantly improved learning performance by creating new episodes from failed ones.
• The proposed DDPG-based method with HER demonstrated successful autonomous driving, mitigating sparse reward problems.
• The method's efficacy was validated across two distinct reward systems and through actual experimental results.

Conclusions:

• The integration of HER with DDPG effectively addresses sparse reward challenges in mobile robot path planning.
• The proposed method enables successful autonomous navigation for mobile robots in unknown environments.
• This approach offers a viable solution for improving the robustness and efficiency of robot path planning systems.