Learning-based near-optimal tracking control for industrial processes with slow and fast modes

Area of Science:

• Control Engineering
• Artificial Intelligence
• Industrial Process Optimization

Background:

• Singular perturbation systems present control challenges due to disparate time scales and unmeasurable states.
• Optimal tracking control (OTC) is crucial for industrial processes but difficult to achieve with unknown system dynamics.
• Existing reinforcement learning (RL) methods struggle with the complexities of singular perturbation systems.

Purpose of the Study:

• To develop a novel reinforcement learning (RL) approach for solving the optimal tracking control (OTC) problem in singular perturbation systems.
• To address the challenges posed by unknown slow states and differing time scales inherent in these systems.
• To enhance the control performance of industrial processes by accurately tracking desired trajectories.

Main Methods:

• Decomposition of singular perturbed systems using singular perturbation (SP) theory.
• Development of a novel off-policy ridge reinforcement learning (RL) algorithm.
• Mathematical manipulation for replacing unmeasured slow states within the RL framework.
• Theoretical analysis to ensure the approximate equivalence of subproblem solutions to the overall OTC problem.

Main Results:

• A new off-policy ridge RL method effectively handles singular perturbation systems.
• The proposed method successfully overcomes challenges of unknown states and multi-time scale dynamics.
• Demonstrated ability to find optimal tracking controllers for systems with immeasurable states.
• Validation through application to a mixed separation thickening process (MSTP) and a numerical example.

Conclusions:

• The developed off-policy ridge RL method provides an effective solution for OTC problems in singular perturbation systems.
• The approach successfully integrates singular perturbation theory and reinforcement learning for complex industrial applications.
• This work offers a significant advancement in controlling industrial processes with challenging dynamic characteristics.