Novel data-driven two-dimensional Q-learning for optimal tracking control of batch process with unknown dynamics.

Area of Science:

• Control Engineering
• Artificial Intelligence
• Chemical Engineering

Background:

• Traditional control methods for batch processes often require accurate system models, which are difficult to obtain for processes with unknown dynamics.
• Practical batch processes present challenges for model-based control due to inherent complexities and unmodeled dynamics.

Purpose of the Study:

• To develop a novel data-driven, model-free control law for optimal tracking control (OTC) in batch processes.
• To address the limitations of existing methods that rely heavily on process models.

Main Methods:

• A data-driven two-dimensional (2D) off-policy Q-learning algorithm was proposed.
• An extended state space equation incorporating state and output error was established.
• A Bellman equation independent of model parameters was derived using 2D value and Q-functions.
• Policy iteration was performed using measured data along batch and time directions.

Main Results:

• The proposed 2D off-policy Q-learning algorithm provides a model-free control law for batch processes.
• The unbiasedness and convergence of the algorithm were mathematically proven.
• Simulations demonstrated improved control and tracking effects with increasing batch numbers.

Conclusions:

• The novel 2D off-policy Q-learning approach effectively achieves optimal tracking control in batch processes without requiring explicit dynamic models.
• The method shows promise for practical applications, as evidenced by improved performance in an injection molding process simulation.