Event-Driven Off-Policy Reinforcement Learning for Control of Interconnected Systems.

Area of Science:

• Control Systems Engineering
• Artificial Intelligence
• Nonlinear Dynamics

Background:

• Decentralized control of interconnected systems is challenging due to uncertainties and interdependencies.
• Existing methods often struggle with complex nonlinear dynamics and require frequent control updates.

Purpose of the Study:

• To develop a novel approximate optimal decentralized control scheme for uncertain input-affine nonlinear-interconnected systems.
• To reduce redundant control updates using an event-triggering mechanism (ETM).

Main Methods:

• Formulating a noncooperative dynamic game at each subsystem, treating interconnections and triggering errors as adversarial players.
• Employing an event-driven off-policy integral reinforcement learning (OIRL) approach with artificial neural networks (NNs) to approximate the Hamilton-Jacobi-Isaac (HJI) equation.
• Designing control policies and ETM thresholds based on the NN-approximated HJI solution.

Main Results:

• Successfully learned approximate solutions to the HJI equation using OIRL and NNs.
• Guaranteed Zeno-free behavior for the event-triggering mechanisms.
• Derived sufficient conditions for uniform ultimate bounded regulation of system states.

Conclusions:

• The proposed framework effectively manages uncertain nonlinear-interconnected systems with decentralized control.
• The combination of game theory, reinforcement learning, and event-triggering offers an efficient approach to optimize control performance and reduce communication overhead.