Symmetry reduction for deep reinforcement learning active control of chaotic spatiotemporal dynamics

Area of Science:

• Fluid dynamics
• Control theory
• Machine learning

Background:

• Deep reinforcement learning (RL) shows promise for complex flow control.
• Neglecting system symmetries hinders naive deep RL performance.
• High-dimensional systems often possess inherent symmetries.

Purpose of the Study:

• To investigate the impact of symmetry reduction on deep RL for flow control.
• To enhance the data efficiency and efficacy of deep RL control policies.
• To demonstrate the robustness of symmetry-aware RL methods.

Main Methods:

• Utilized the Kuramoto-Sivashinsky equation (KSE) as a testbed.
• Implemented deep reinforcement learning in a symmetry-reduced state space.
• Applied equally spaced actuators for control.
• Focused on minimizing system dissipation and power cost.

Main Results:

• Symmetry-reduced deep RL significantly improved data efficiency and policy efficacy compared to naive deep RL.
• The learned policy autonomously discovered and stabilized a system equilibrium state.
• The control policy demonstrated robustness against noise and unobserved system parameters.

Conclusions:

• Incorporating symmetry awareness into deep RL is crucial for effective flow control.
• Symmetry reduction alleviates limitations of naive deep RL in high-dimensional systems.
• Symmetry-aware deep RL offers a robust and efficient approach for complex control tasks.