Zermelo's problem: Optimal point-to-point navigation in 2D turbulent flows using reinforcement learning

Area of Science:

• Fluid dynamics
• Robotics
• Artificial Intelligence

Background:

• Zermelo's problem addresses time-minimal navigation in fluid flow.
• Vessels with slip velocity in turbulent seas present complex navigation challenges.

Purpose of the Study:

• Investigate Reinforcement Learning (RL) for solving Zermelo's problem.
• Compare RL with traditional Optimal Navigation (ON) methods for vessel navigation in 2D turbulent seas.

Main Methods:

• Utilized an Actor-Critic RL algorithm.
• Applied RL to time-independent and chaotically evolving flow configurations.
• Compared RL results with analytical ON protocols for a frozen flow case.

Main Results:

• RL algorithm successfully found quasi-optimal navigation paths.
• RL solutions demonstrated superior robustness against initial condition changes and external noise compared to ON.
• RL effectively leveraged flow properties for navigation, particularly at low steering speeds.

Conclusions:

• RL offers a practical and robust solution for Zermelo's problem in turbulent environments.
• Traditional ON methods are unstable and less suitable for real-world vessel navigation in complex flows.
• RL's adaptability makes it a promising approach for autonomous navigation in dynamic fluid environments.