Discrete-time online learning control for a class of unknown nonaffine nonlinear systems using reinforcement learning

Area of Science:

• Control Theory
• Machine Learning
• Nonlinear Systems

Background:

• Discrete-time (DT) nonlinear systems often face challenges with unknown bounded disturbances.
• Achieving desired tracking performance in such systems requires robust adaptive control strategies.

Purpose of the Study:

• To develop a reinforcement-learning-based direct adaptive control strategy.
• To address multi-input-multi-output (MIMO) unknown nonaffine nonlinear DT systems.
• To ensure uniform ultimate boundedness of tracking errors and neural network weights.

Main Methods:

• Utilized two neural networks (NNs): an action NN for control signal generation and nonlinearity cancellation via feedback linearization, and a critic NN for cost function estimation.
• Employed the Implicit Function Theorem and heuristic dynamic programming.
• Implemented online weight updates for both NNs.
• Applied Lyapunov's direct method for stability analysis.

Main Results:

• The proposed direct adaptive control method effectively manages unknown bounded disturbances in nonlinear DT systems.
• Demonstrated uniform ultimate boundedness of closed-loop tracking errors and estimated neural network weights.
• Validated the approach's effectiveness through two numerical examples.

Conclusions:

• The reinforcement-learning-based direct adaptive control offers a viable solution for tracking control in complex nonlinear systems.
• Online adaptation of neural network weights ensures system stability and performance robustness.
• The method shows promise for practical applications involving uncertain dynamic systems.