Control of chaotic systems by deep reinforcement learning

Area of Science:

• Nonlinear dynamics
• Control theory
• Artificial intelligence

Background:

• The Kuramoto-Sivashinsky (KS) equation models complex spatiotemporal dynamics.
• Controlling chaotic systems is challenging due to their sensitivity to initial conditions.
• Deep reinforcement learning (DRL) has shown promise in complex control tasks.

Purpose of the Study:

• To apply Deep Reinforcement Learning (DRL) for controlling the nonlinear, chaotic Kuramoto-Sivashinsky (KS) system.
• To investigate the efficacy of DRL with restricted actuation and partial state knowledge.
• To demonstrate the stabilization of unstable fixed points in the KS system.

Main Methods:

• Utilized model-free Deep Reinforcement Learning (DRL) controllers.
• Employed deep neural networks for value function and policy approximation.
• Implemented restricted localized actuation and partial state measurements.

Main Results:

• DRL successfully stabilized the chaotic dynamics of the KS system around target states.
• Controllers demonstrated robustness across various initial conditions and trajectories.
• Achieved stabilization using only local measurements, indicating model-free DRL's potential.

Conclusions:

• DRL offers a powerful, robust method for controlling complex nonlinear and chaotic systems.
• The approach's reliance on local measurements suggests broader applicability in fluid dynamics and turbulence control.
• This work highlights DRL's capability to achieve precise control even with limited system information.