Data-driven model reference control of MIMO vertical tank systems with model-free VRFT and Q-Learning

Area of Science:

• Control Engineering
• Machine Learning
• Nonlinear Systems

Background:

• Model-free control is challenging for unknown nonlinear systems.
• Efficient state-action space exploration is crucial for Q-learning convergence.
• Virtual Reference Feedback Tuning (VRFT) can provide initial stabilizing controllers.

Purpose of the Study:

• To develop a combined VRFT-Q-learning approach for tuning nonlinear static state feedback controllers.
• To achieve optimal model reference tracking for unknown nonlinear systems.
• To improve learning convergence and controller performance.

Main Methods:

• A novel iterative Batch Fitted Q-learning strategy using actor-critic neural networks.
• Integration of VRFT to gather initial data and guide Q-learning exploration.
• Application of the mixed VRFT-Batch Fitted Q-learning approach for controller tuning.

Main Results:

• The proposed method successfully tunes nonlinear controllers for model reference tracking.
• Experimental validation on a nonlinear coupled two-tank system demonstrates effectiveness.
• The combined approach overcomes limitations of individual methods for exploration and learning.

Conclusions:

• The mixed VRFT-Batch Fitted Q-learning approach offers a robust solution for model-free control of nonlinear systems.
• This method enhances learning efficiency and achieves superior tracking performance.
• It provides a practical framework for real-world control applications.