Networked controller and observer design of discrete-time systems with inaccurate model parameters

Area of Science:

• Control Systems Engineering
• Machine Learning
• Networked Systems

Background:

• Networked control systems (NCS) face challenges like inaccurate models, unmeasurable states, and network delays.
• Traditional control methods struggle with these uncertainties, hindering optimal performance.

Purpose of the Study:

• To develop a novel off-policy Q-learning method for optimal observer gain and controller design in NCS.
• To address systems with inaccurate models, unmeasurable states, and network-induced delays using only measured data.

Main Methods:

• Formulated an optimization problem for NCS including a plant, state observer, and Smith predictor.
• Employed a Smith predictor to compensate for network delays and ensure separation principle.
• Implemented an off-policy Q-learning algorithm to learn optimal observer gain and controller from input, output, and delayed states.

Main Results:

• Derived a novel off-policy Q-learning algorithm that does not require system matrices.
• Rigorously proved the convergence of iterative observer gain and controller gain.
• Simulation results verified the effectiveness of the proposed method.

Conclusions:

• The proposed off-policy Q-learning method effectively optimizes NCS with model uncertainties and network delays.
• The method successfully learns optimal control policies using only available system measurements.
• This approach offers a robust solution for practical NCS applications.