Optimized Backstepping Tracking Control Using Reinforcement Learning for a Class of Stochastic Nonlinear...

Area of Science:

• Control Systems Engineering
• Machine Learning
• Nonlinear Dynamics

Background:

• Stochastic nonlinear strict-feedback systems present significant control challenges due to unknown dynamics and inherent stochastic disturbances.
• Existing reinforcement learning (RL) methods for such systems are often complex and require specific conditions like persistence excitation and known system dynamics, limiting their practical application.

Purpose of the Study:

• To propose a novel optimized backstepping (OB) control scheme for stochastic nonlinear strict-feedback systems with unknown dynamics.
• To develop a simplified reinforcement learning (RL) strategy that overcomes the limitations of existing methods, enhancing control performance and applicability.

Main Methods:

• The study employs an identifier-critic-actor architecture for reinforcement learning (RL).
• Virtual and actual controls in the backstepping design are optimized as solutions to subsystem problems.
• Critic and actor updating laws are derived from the negative gradient of a simplified function based on the Hamilton-Jacobi-Bellman (HJB) equation, reducing algorithmic complexity.

Main Results:

• The proposed RL algorithm significantly simplifies the control design process.
• The method successfully eliminates the need for persistence excitation and knowledge of system dynamics.
• Both theoretical analysis and simulations confirm the achievement of desired system performance.

Conclusions:

• The optimized backstepping control scheme offers an effective and simplified approach for controlling stochastic nonlinear systems.
• The developed RL strategy provides a more practical and broadly applicable solution compared to existing methods.
• This work represents a significant advancement in the field of stochastic optimization control.