Comparing Deep Reinforcement Learning Algorithms' Ability to Safely Navigate Challenging Waters.

Area of Science:

• Robotics
• Artificial Intelligence
• Control Systems

Background:

• Reinforcement Learning (RL) controllers are effective for path following and collision avoidance.
• Optimizing RL algorithm setups for these dual objectives remains challenging.
• Underactuated surface vehicles require sophisticated control strategies.

Purpose of the Study:

• To develop a methodology for analyzing RL algorithm performance in path following and collision avoidance.
• To compare various RL algorithms in increasingly complex environments.
• To identify optimal RL configurations for underactuated surface vehicles.

Main Methods:

• Applied a range of RL algorithms to path-following and collision-avoidance tasks.
• Analyzed algorithm performance and task-specific behaviors.
• Utilized environments of increasing complexity and varied reward functions.
• Investigated generalization capabilities with domain gaps and sensor suite dimensionality reduction.

Main Results:

• Proximal Policy Optimization (PPO) demonstrated superior robustness to environmental complexity and reward function changes.
• PPO generalized effectively to environments with significant domain gaps.
• The proposed reward function improved competing algorithms' performance in the training environment.
• Sensor dimensionality reduction combined with domain gap impaired generalization for non-PPO algorithms.

Conclusions:

• PPO offers a robust solution for path following and collision avoidance in underactuated surface vehicles.
• Sensor suite design and domain gap management are critical for RL generalization.
• Further research can refine RL strategies for enhanced autonomous navigation.