Synergetic Learning Neuro-Control for Unknown Affine Nonlinear Systems With Asymptotic Stability Guarantees

Area of Science:

• Control Systems Engineering
• Machine Learning
• Nonlinear Dynamics

Background:

• Optimal control for unknown nonlinear systems is challenging.
• Conventional methods often require complete system dynamics knowledge.
• Reinforcement learning offers a potential model-free solution.

Purpose of the Study:

• Develop a synergetic learning algorithm (SLA) for optimal control of unknown affine nonlinear systems.
• Establish a model-free Hamilton-Jacobi-Bellman equation (MF-HJBE) using off-policy reinforcement learning.
• Demonstrate asymptotic stability and cost function optimization using the developed SLA.

Main Methods:

• Deduction of a model-free HJBE (MF-HJBE) via off-policy reinforcement learning.
• Bridging the equivalence between HJBE and MF-HJBE based on solution uniqueness.
• Utilizing a two-agent synergetic learning (SL) system (critic and actor agents) with an experience replay (ER)-based learning rule.

Main Results:

• The MF-HJBE solution, when it exists, guarantees asymptotic system stability and optimal cost function.
• The critic agent evolves towards the optimal cost function.
• The actor agent evolves towards the optimal control and ensures system asymptotic stability.

Conclusions:

• The developed SLA effectively learns optimal control for unknown affine nonlinear systems.
• The model-free approach using RL provides a robust alternative to traditional methods.
• Simulations confirm the feasibility and effectiveness of the SLA for complex systems.