Deep Reinforcement Learning for Autonomous Underwater Navigation: A Comparative Study with DWA and Digital Twin...

Area of Science:

• Robotics
• Artificial Intelligence
• Marine Technology

Background:

• Autonomous underwater navigation faces challenges like GPS denial, poor visibility, and obstacles.
• The BlueROV2 platform is utilized for scientific investigation in these demanding conditions.

Purpose of the Study:

• To develop and evaluate a deep reinforcement learning approach for autonomous underwater navigation.
• To compare the proposed method against the Dynamic Window Approach (DWA) for obstacle avoidance.

Main Methods:

• A Proximal Policy Optimization (PPO) algorithm was employed, integrating target navigation, a virtual occupancy grid, and raycasting for situational awareness.
• The agent's observation space was designed as a high-dimensional (84-dimensional) input.
• Performance was evaluated in simulation and on a physical BlueROV2 using a 3D digital twin.

Main Results:

• The PPO-based policy significantly outperformed the DWA planner in cluttered underwater environments.
• The PPO agent exhibited enhanced local adaptation and a reduction in collisions.
• Successful transfer of learned navigation behaviors from simulation to the real world was demonstrated.

Conclusions:

• Deep reinforcement learning, specifically PPO, is a viable and effective method for autonomous underwater navigation.
• The proposed observation space design provides comprehensive situational awareness for the navigation agent.
• The study validates the transferability of simulated DRL policies to physical robotic systems.