Learning obstacle avoidance and predation in complex reef environments with deep reinforcement learning.

Area of Science:

• Marine Biology
• Computational Fluid Dynamics
• Artificial Intelligence

Background:

• Reef ecosystems are crucial habitats for fish, influencing their behavior, feeding, and survival.
• Understanding fish navigation in complex, dynamic water flow is vital for ecology and biomimetics.
• Simulating fish interactions with moving targets and obstacles in reef environments presents significant challenges.

Purpose of the Study:

• To investigate fish predation and foraging behaviors in intricate reef environments.
• To develop an integrated simulation framework combining AI and fluid dynamics.
• To analyze the impact of reward mechanisms on simulated fish decision-making.

Main Methods:

• Utilized a simulation framework integrating deep reinforcement learning (DRL) with the immersed boundary lattice Boltzmann method (lB-LBM) for fluid-structure interaction.
• Employed the Soft Actor-Critic (SAC) algorithm to enhance exploration and address sparse reward challenges.
• Developed a tailored reward shaping method to effectively capture outcomes and trend characteristics.

Main Results:

• Demonstrated the convergence and robustness of the integrated simulation framework through two case studies.
• Successfully simulated fish capturing randomly moving targets in hydrostatic flow.
• Modeled fish counter-current foraging in reef environments to capture drifting food.

Conclusions:

• The developed simulation framework effectively models complex fish behaviors in dynamic aquatic environments.
• Reward shaping significantly influences the decision-making processes of simulated fish.
• This research provides insights into fish ecology, behavior, and offers potential for biomimetic applications.