Related Experiment Video
Updated: Jun 19, 2026

07:47
Shallow Water Paddling Variants of Water Maze Tests in Mice
Published on: June 3, 2013
24.2K
A critical assessment of reinforcement learning methods for microswimmer navigation in complex flows
Selim Mecanna1, Aurore Loisy2, Christophe Eloy3
1Aix Marseille Univ, CNRS, Centrale Med, IRPHE, Marseille, France.
The European Physical Journal. E, Soft Matter
|October 3, 2025
Summary
Reinforcement learning for autonomous navigation in fluid flows shows that advanced algorithms like Proximal Policy Optimization (PPO) significantly outperform simpler methods, achieving near-optimal strategies in complex environments.
Area of Science:
- Fluid Mechanics
- Robotics
- Artificial Intelligence
Background:
- Planktonic organisms and autonomous robots face challenges navigating fluid flows using only onboard sensors.
- Reinforcement learning (RL) is increasingly used for this, but algorithm performance is often unclear.
Purpose of the Study:
- To quantitatively assess RL methods for navigation in partially observable fluid flows.
- To compare the performance of different RL algorithms against a known quasi-optimal policy.
Main Methods:
- Introduced a directional navigation problem with a known analytical solution.
- Evaluated Q-Learning and Advantage Actor Critic (A2C) against Proximal Policy Optimization (PPO).
- Utilized vectorized environments, generalized advantage estimation, and hyperparameter optimization for PPO.
Main Results:
- Common RL algorithms (Q-Learning, A2C) showed poor performance and robustness in various flows.
- PPO significantly surpassed these methods, matching theoretical quasi-optimal performance in turbulent flow.
- PPO demonstrated robust and near-optimal navigation strategies.
Conclusions:
- Algorithm selection and implementation details are crucial for effective autonomous navigation in complex flows.
- Advanced RL algorithms like PPO are essential for developing sophisticated navigation strategies.
- Fine-tuning and advanced techniques are key to achieving high performance in RL-based navigation.

