Moment-Based Reinforcement Learning for Ensemble Control.

Area of Science:

• Systems Science
• Control Engineering
• Dynamical Systems

Background:

• Controlling collective behavior in populations of dynamical systems (ensemble control) is challenging due to under-actuation and limited monitoring.
• Mathematical models for ensemble systems are often unknown, necessitating robust control strategies.
• Existing methods struggle with heterogeneity and scalability in large-scale systems.

Purpose of the Study:

• To develop a data-driven control framework for steering dynamic populations using reinforcement learning (RL).
• To address the challenge of controlling ensemble systems with limited, population-level data.
• To robustly compensate for system heterogeneity in broadcast control scenarios.

Main Methods:

• Proposed a reinforcement learning (RL)-based framework utilizing population-level aggregated measurements.
• Introduced and derived an 'ensemble moment system' from aggregated data.
• Learned optimal control policies using RL on the derived moment system for linear, bilinear, and nonlinear systems.

Main Results:

• The proposed moment-based RL approach successfully steered dynamic populations towards desired behaviors.
• Demonstrated feasibility and scalability across diverse ensemble systems (linear, bilinear, nonlinear).
• Achieved significantly better averaged-reward compared to three existing control methods.

Conclusions:

• The novel moment-based RL framework offers an effective solution for data-driven ensemble control.
• The approach robustly compensates for system heterogeneity using aggregated measurements.
• This method provides a scalable and high-performing alternative for complex dynamical population control.