NN-Based Reinforcement Learning Optimal Control for Inequality-Constrained Nonlinear Discrete-Time Systems With...

Area of Science:

• Control Theory
• Artificial Intelligence
• Nonlinear Systems

Background:

• Constrained control problems in affine nonlinear discrete-time systems are challenging due to system dynamics and external disturbances.
• Existing methods may struggle with complex constraints and uncertainties, necessitating advanced control strategies.
• Actor-critic neural networks (NNs) offer a promising framework for adaptive and optimal control.

Purpose of the Study:

• To develop an optimal controller for affine nonlinear discrete-time systems subject to constraints and disturbances.
• To leverage actor-critic NNs for generating control signals and evaluating controller performance.
• To ensure the stability and bounded performance of the control system under adverse conditions.

Main Methods:

• Utilized actor-NNs for control signal generation and critic-NNs for performance evaluation.
• Transformed constrained optimal control into an unconstrained problem by incorporating penalty functions into the cost function.
• Employed Game theory to determine the optimal control input considering worst-case disturbances and Lyapunov stability theory for performance guarantees.

Main Results:

• The proposed actor-critic NN-based controller effectively addresses constrained control problems in nonlinear discrete-time systems.
• The controller ensures that control signals are uniformly ultimately bounded (UUB), guaranteeing system stability.
• Numerical simulations on a third-order dynamic system validated the effectiveness of the developed control algorithms.

Conclusions:

• The actor-critic NN approach provides a robust solution for optimal control of constrained nonlinear systems with disturbances.
• The integration of Game theory and Lyapunov stability theory enhances the reliability and performance guarantees of the controller.
• The proposed method demonstrates significant potential for practical applications requiring precise and stable control.