Reinforcement Learning-Based Adaptive Optimal Control for Nonlinear Systems With Asymmetric Hysteresis

Area of Science:

• Control Systems Engineering
• Nonlinear Dynamics
• Machine Learning

Background:

• Nonlinear affine systems often exhibit complex behaviors like hysteresis, particularly in actuators.
• Prandtl-Ishlinskii (PI) hysteresis introduces significant challenges in achieving precise system tracking and optimal control.
• Uncertainties in hysteresis models complicate the direct application of standard control and optimization techniques.

Purpose of the Study:

• To investigate the adaptive optimal tracking problem for nonlinear affine systems affected by asymmetric PI hysteresis.
• To develop a robust control scheme that overcomes the difficulties posed by hysteresis in actuator dynamics.
• To enable the application of actor-critic (A-C) learning mechanisms for optimal control in the presence of hysteresis.

Main Methods:

• Development of an inverse model to compensate for PI hysteresis using a shift factor.
• Design of a feedback controller that integrates estimation and approximation errors into the Hamilton function's error term.
• Derivation of optimal control laws via partial derivatives of the Hamiltonian function after nonlinearity compensation.

Main Results:

• Successfully compensated for asymmetric Prandtl-Ishlinskii hysteresis nonlinearities in actuator models.
• Enabled the construction of Hamilton functions despite hysteresis-induced input delays.
• Achieved adaptive optimal tracking for the targeted class of nonlinear systems through simulation validation.

Conclusions:

• The proposed actor-critic learning-based scheme effectively addresses the adaptive optimal tracking problem in systems with PI hysteresis.
• The inverse model compensation strategy successfully mitigates the impact of hysteresis on control system performance.
• The developed method provides a viable approach for optimal control design in complex nonlinear systems with actuator nonlinearities.