Multi-agent Reinforcement Learning for the Control of Three-Dimensional Rayleigh-Bénard Convection.

Area of Science:

• Fluid Dynamics
• Computational Physics
• Artificial Intelligence

Background:

• Deep reinforcement learning (DRL) is increasingly applied to flow control challenges.
• Multi-agent reinforcement learning (MARL), a subset of DRL, excels in controlling flows with local and invariant properties.
• Three-dimensional Rayleigh-Bénard convection (RBC) presents complex flow dynamics amenable to advanced control strategies.

Purpose of the Study:

• To implement and evaluate a MARL-based control system for three-dimensional Rayleigh-Bénard convection.
• To assess the effectiveness of MARL in reducing convection intensity and stabilizing flow patterns.
• To compare MARL performance against traditional proportional control methods.

Main Methods:

• MARL agents were developed to control temperature distribution on segmented bottom walls in RBC simulations.
• Simulations were conducted for two RBC regimes at Rayleigh numbers of 500 and 750.
• Learned MARL policies were evaluated for convection intensity reduction, pattern transformation, and transferability to larger domains.

Main Results:

• MARL control reduced convection intensity by 23.5% at Ra=500 and 8.7% at Ra=750.
• Irregular convective patterns were transformed into stable, straight rolls, indicating a more stable flow regime.
• MARL significantly outperformed proportional control and demonstrated successful policy transfer to a larger simulation domain.

Conclusions:

• MARL provides a powerful and effective method for controlling complex fluid phenomena like RBC.
• The learned MARL strategy exhibits robustness and adaptability, outperforming conventional control techniques.
• The invariant properties of MARL enable direct transfer of learned policies, enhancing practical applicability.