Optimal Output Regulation of Linear Discrete-Time Systems With Unknown Dynamics Using Reinforcement Learning

Area of Science:

• Control Systems Engineering
• Reinforcement Learning
• Optimization Theory

Background:

• Output regulation is crucial for discrete-time systems operating under external disturbances.
• Traditional methods often require precise knowledge of system dynamics, which is not always available.
• Model-free approaches offer a promising alternative for complex control scenarios.

Purpose of the Study:

• To develop a model-free optimal control strategy for discrete-time systems facing disturbances.
• To address the output regulation problem without prior knowledge of system dynamics.
• To decompose the problem into static and dynamic optimization sub-problems.

Main Methods:

• A reinforcement learning-based, model-free, off-policy algorithm is proposed.
• The approach breaks down the problem into static and dynamic optimization tasks.
• Measured data is utilized to solve the dynamic optimization problem, followed by the static problem.

Main Results:

• The presented algorithm successfully solves the output regulation problem without system identification.
• The method is robust to probing noise required for persistence of excitation.
• Simulation results validate the effectiveness of the model-free optimal control strategy.

Conclusions:

• A novel model-free reinforcement learning framework is established for optimal output regulation.
• The proposed method enhances control system design by eliminating the need for system dynamics models.
• This approach offers a practical solution for real-world discrete-time systems with disturbances.