Deep reinforcement learning for active flow control in a turbulent separation bubble.

Area of Science:

• Fluid Dynamics
• Computational Fluid Dynamics
• Machine Learning

Background:

• Turbulent separation bubbles (TSB) present significant challenges in fluid dynamics.
• Classical periodic forcing offers limited control efficacy for TSBs.

Purpose of the Study:

• To numerically assess the control efficacy of deep reinforcement learning (DRL) against classical periodic forcing for TSBs.
• To investigate the feasibility of applying DRL control strategies trained on coarse grids to fine grids, reducing computational costs.

Main Methods:

• Numerical simulation of a turbulent separation bubble.
• Comparison of deep reinforcement learning (DRL) control with periodic forcing.
• Assessment of DRL strategy transferability from coarse to fine computational grids.
• Analysis of flow physics and vortex dynamics induced by DRL control.

Main Results:

• DRL-based control reduced TSB area by 9.0%, surpassing periodic control's 6.8% reduction.
• A DRL strategy trained on a coarse grid effectively controlled flow on a fine grid.
• DRL provided smoother control and instantaneous momentum conservation.
• DRL induced large-scale counter-rotating vortices across a range of frequencies.

Conclusions:

• DRL offers superior control efficacy for turbulent separation bubbles compared to periodic forcing.
• Coarse-grid DRL training is a viable method to reduce computational expense in turbulent flow control.
• The developed open-source CFD and DRL framework supports exascale computing for advanced fluid dynamics research.