Optimal Control for Constrained Discrete-Time Nonlinear Systems Based on Safe Reinforcement Learning.

Area of Science:

• Control Engineering
• Artificial Intelligence
• Nonlinear System Dynamics

Background:

• Optimal control of nonlinear systems is challenged by state and input constraints.
• Traditional reinforcement learning (RL) methods using quadratic utility functions struggle with these constraints.

Purpose of the Study:

• Develop a novel optimal control approach for constrained discrete-time (DT) nonlinear systems using safe RL.
• Address the limitations of existing methods in handling complex system constraints.

Main Methods:

• Introduce a barrier function (BF) integrated with the value function to convert constrained problems into unconstrained ones.
• Develop a constrained policy iteration (PI) algorithm utilizing two neural networks (NNs) for policy and value function approximation.

Main Results:

• The proposed method effectively transforms constrained optimization into an unconstrained problem with a guaranteed minimum at the origin.
• The constrained PI algorithm successfully satisfies state and input constraints for nonlinear systems.
• The approach preserves the convergence and optimality properties of traditional PI algorithms.

Conclusions:

• The novel safe RL approach with barrier functions provides an effective solution for optimal control of constrained DT nonlinear systems.
• The use of neural networks facilitates the derivation of constrained optimal control policies.
• Simulation results demonstrate the practical effectiveness of the proposed method.