Model-Free Optimal Tracking Control of Nonlinear Input-Affine Discrete-Time Systems via an Iterative Deterministic...

Area of Science:

• Control Engineering
• Machine Learning
• Artificial Intelligence

Background:

• Optimal tracking control (OTC) is crucial for unknown nonlinear discrete-time (DT) systems.
• Existing methods like discount factor-based Q-learning (DFQL) have limitations in handling complex system dynamics.

Purpose of the Study:

• To propose a novel model-free dynamic inversion-based Q-learning (DIQL) algorithm for OTC problems.
• To ensure the algorithm is model-free, off-policy, and eliminates tracking errors.

Main Methods:

• Developed a new deterministic Q-learning iterative scheme for improved data utilization and computational efficiency.
• Designed a model-based off-policy DIQL algorithm and analyzed its convergence and stability.
• Introduced neural networks (NNs) to create a model-free version of the DIQL algorithm.

Main Results:

• The proposed DIQL algorithm effectively eliminates tracking errors in unknown nonlinear input-affine DT systems.
• The algorithm demonstrates improved data utilization and computational resource savings.
• Convergence and stability analyses confirm the algorithm's robustness, even with probing noise.

Conclusions:

• The novel model-free DIQL algorithm provides an effective solution for OTC problems in unknown systems.
• The approach enhances efficiency and accuracy compared to existing methods.
• The use of NNs makes the algorithm applicable without prior system dynamics knowledge.