Plume Tracing via Model-Free Reinforcement Learning Method.

Area of Science:

• Robotics
• Oceanography
• Artificial Intelligence

Background:

• Deep-sea exploration faces challenges due to turbulent environments.
• Autonomous Underwater Vehicles (AUVs) require effective strategies for tasks like plume tracing.
• Environmental dynamics necessitate adaptive navigation methods.

Purpose of the Study:

• To develop and validate an advanced plume-tracing strategy for AUVs in deep-sea turbulent conditions.
• To model the plume-tracing problem as a partially observable Markov decision process.
• To enhance AUV navigation and efficiency in complex marine settings.

Main Methods:

• A long short-term memory-based reinforcement learning framework was developed.
• Continuous temporal difference and deterministic policy gradient methods were employed.
• A supervised strategy from dynamic programming was integrated as prior knowledge.

Main Results:

• The proposed framework generated a smooth and effective plume-tracing strategy.
• Simulation experiments based on Reynolds-averaged Navier-Stokes equations validated the strategy's performance.
• The integration of historical data and exploration technology improved algorithm efficiency.

Conclusions:

• The developed reinforcement learning framework offers a robust solution for AUV plume tracing in challenging environments.
• The strategy demonstrates significant potential for enhancing autonomous deep-sea exploration.
• This research contributes to advancing AUV capabilities in dynamic oceanic settings.