Efficient reinforcement learning with partial observables for fluid flow control.

Area of Science:

• Fluid dynamics
• Control theory
• Machine learning

Background:

• Reinforcement learning (RL) typically requires extensive system observations for active control problems.
• Viscous flow systems, despite potentially low-dimensional dynamics, necessitate numerous observables in standard RL frameworks.
• Partial observability in RL is challenging, often requiring full system state information.

Purpose of the Study:

• To develop a consistent reinforcement learning algorithm for active flow control under partial observability.
• To leverage the low-dimensional properties of viscous flow systems for improved RL efficiency.
• To demonstrate a more stable and efficient RL approach compared to existing methods in flow control.

Main Methods:

• Constructed a novel RL algorithm tailored for partially observable viscous flow systems.
• Utilized the inherent low-dimensional state-space characteristics of viscous flows.
• Tested the algorithm on typical active flow control scenarios.

Main Results:

• The proposed RL algorithm demonstrated enhanced stability and efficiency.
• The algorithm performed effectively even with a limited number of system observables.
• Outperformed existing RL algorithms in active flow control tasks.

Conclusions:

• The developed RL algorithm successfully addresses partial observability in viscous flow control.
• Leveraging low-dimensional system properties enables efficient and stable flow control with minimal observables.
• This approach offers a promising alternative for complex fluid dynamics control problems.