Event-Triggered Control of Nonlinear Discrete-Time System With Unknown Dynamics Based on HDP(λ)

Area of Science:

• Control Systems Engineering
• Artificial Intelligence
• Nonlinear Dynamics

Background:

• Traditional heuristic dynamic programming (HDP) methods accelerate learning but face computational challenges with increasing iterations.
• High computational complexity hinders optimal control in systems with limited resources.
• Existing value-gradient learning methods can be computationally intensive due to state-associated variables.

Purpose of the Study:

• To propose an event-triggered HDP (ETHDP) optimal control strategy for nonlinear discrete-time (NDT) systems with unknown dynamics.
• To reduce computation and communication requirements while ensuring system stability.
• To address the challenges of high computational costs in iterative optimal control processes.

Main Methods:

• Derivation of the iterative relation for the lambda-return of the final target value.
• Design of an event-triggered condition to ensure system stability and reduce resource demands.
• Development of a model-actor-critic neural network (NN) architecture for state evaluation, lambda-return calculation, and real-time update error determination.
• Approximation of the event-triggered optimal control signal and one-step-return value using actor and critic NNs.
• Application of Lyapunov technology to demonstrate uniformly ultimately bounded (UUB) stability for system states and NN weight errors.

Main Results:

• The proposed ETHDP (lambda) strategy effectively reduces computation and communication overhead.
• System stability and NN weight errors are proven to be uniformly ultimately bounded (UUB) using Lyapunov analysis.
• The strategy demonstrates effectiveness in nonlinear discrete-time systems with unknown dynamics.
• The model-actor-critic NN structure successfully approximates control signals and return values.

Conclusions:

• The event-triggered HDP (ETHDP) (lambda) strategy offers an efficient approach to optimal control for NDT systems.
• The method successfully balances performance with reduced computational and communication burdens.
• The theoretical stability guarantees provide confidence in the practical application of the proposed strategy.