A Simulator and First Reinforcement Learning Results for Underwater Mapping.

Area of Science:

• Robotics
• Artificial Intelligence
• Environmental Science

Background:

• Underwater mapping faces challenges due to a lack of accurate sensor behavior models.
• Underwater litter presents a significant environmental problem, addressed by the Horizon 2020 SeaClear project.
• Reinforcement learning is well-suited for robot control in unknown environments but has not been applied to underwater mapping.

Purpose of the Study:

• To apply deep reinforcement learning (DRL) for the first time to the problem of underwater litter mapping.
• To develop a fast, custom simulator for mapping seafloor objects using sonar-like sensors on underwater vehicles.
• To evaluate and improve state-of-the-art DRL algorithms for this specific application.

Main Methods:

• Implementation of two state-of-the-art deep reinforcement learning algorithms.
• Development of a novel, high-speed simulator for underwater object mapping.
• Adaptation and enhancement of DRL algorithms for mapping-specific challenges.
• Extensive numerical experiments comparing DRL variants and a baseline trajectory.

Main Results:

• The developed DRL approach significantly enhances the speed of underwater litter collection.
• The best-performing DRL method demonstrated superior performance compared to a standard lawn mower trajectory.
• The custom-built simulator enabled efficient training of DRL agents by providing millions of necessary samples.

Conclusions:

• Deep reinforcement learning is a viable and effective approach for autonomous underwater mapping and litter collection.
• The developed simulator is crucial for advancing DRL applications in marine robotics.
• This research offers a promising solution for tackling underwater pollution through robotic systems.