Robust ASV Navigation Through Ground to Water Cross-Domain Deep Reinforcement Learning

Area of Science:

• Robotics
• Artificial Intelligence
• Machine Learning

Background:

• Deep Reinforcement Learning (DRL) faces data sparsity challenges in complex environments.
• Training DRL agents for marine navigation is costly and risky.
• Generalizing DRL agents across different domains remains a significant hurdle.

Purpose of the Study:

• To present a novel framework for DRL agent training and vehicle integration.
• To overcome data sparsity in challenging domains like marine environments.
• To enable DRL agents to generalize from accessible domains to inaccessible ones with minimal retraining.

Main Methods:

• A DRL agent training and vehicle integration methodology is developed.
• Leveraging accessible domains (e.g., ground) for initial agent training.
• Integrating DRL at a high level of vehicle control, separated from vehicle dynamics and environmental constraints.
• Utilizing an autonomy package with a tertiary multilevel controller for DRL agent interfacing.

Main Results:

• A Deep Q Network (DQN) agent trained for obstacle avoidance in a simulated ground environment demonstrated generalization capabilities.
• Experimental validation included simulated water environments and real-world robotic platforms (ground and water).
• The methodology successfully enabled the development of marine DRL agents for Autonomous Surface Vehicle (ASV) navigation by leveraging ground-based training.

Conclusions:

• Accessible, data-rich domains can be effectively used to train DRL agents for marine navigation.
• This approach facilitates rapid, iterative agent development without risking ASV loss or incurring high deployment costs.
• Landlocked institutions can now develop advanced marine DRL agents for ASV applications.