Reinforcement Learning-Based Model Predictive Control for Discrete-Time Systems.

Area of Science:

• Control Systems Engineering
• Artificial Intelligence
• Machine Learning

Background:

• Traditional model predictive control (MPC) requires complex offline design for terminal cost, auxiliary controllers, and terminal constraints.
• These design requirements can limit flexibility and increase computational burden, particularly for nonlinear systems.

Purpose of the Study:

• To propose a novel reinforcement learning-based model predictive control (RLMPC) scheme for discrete-time systems.
• To integrate model predictive control (MPC) and reinforcement learning (RL) through policy iteration (PI) for improved control policy generation and evaluation.
• To eliminate the need for offline design of terminal cost, auxiliary controller, and terminal constraint in MPC, thereby enhancing flexibility and reducing computational load.

Main Methods:

• The proposed RLMPC scheme utilizes MPC as a policy generator and reinforcement learning (RL) to evaluate the generated policy.
• The value function obtained from RL is used as the terminal cost for MPC, iteratively improving the control policy.
• Policy iteration (PI) is employed to integrate MPC and RL.

Main Results:

• The RLMPC scheme successfully eliminates the need for offline design of terminal cost, auxiliary controller, and terminal constraint.
• RLMPC allows for a more flexible prediction horizon, potentially reducing computational burden.
• Rigorous analysis confirmed the convergence, feasibility, and stability properties of the RLMPC scheme.
• Simulations demonstrated that RLMPC achieves performance comparable to traditional MPC for linear systems and superior performance for nonlinear systems.

Conclusions:

• The proposed RLMPC scheme offers a flexible and computationally efficient alternative to traditional MPC.
• RLMPC demonstrates strong control performance, particularly for complex nonlinear systems.
• This novel approach has significant potential for advancing control strategies in various engineering applications.