An Advanced Optimal Tracking Control for Nonlinear Discrete-Time Systems Based on (N + 1)-Step Gradient Learning.

Area of Science:

• Control Systems Engineering
• Machine Learning
• Nonlinear Dynamics

Background:

• Nonlinear discrete-time systems present challenges in achieving fast convergence and zero tracking error.
• Existing N-step gradient learning algorithms often rely on infinite future reward assumptions and quadratic input terms.

Purpose of the Study:

• To develop an advanced optimal control method for nonlinear discrete-time systems.
• To accelerate convergence performance and eliminate tracking errors.
• To introduce a novel tracking error index independent of the discount factor.

Main Methods:

• An improved (N+1)-step gradient learning algorithm is proposed.
• A novel tracking error index is introduced, avoiding quadratic input terms.
• Value iteration (VI) and policy iteration (PI) methods are used to analyze convergence and stability.
• An actor-critic structure with four neural networks is employed for implementation.

Main Results:

• The proposed algorithm achieves faster convergence and eliminates tracking errors.
• The method obtains optimal control policies without calculating reference control input.
• Convergence, monotonicity, optimality, and stability properties are derived without zero initial function assumptions.
• Simulations on a helicopter system demonstrate the method's efficacy.

Conclusions:

• The advanced optimal control method effectively addresses nonlinear optimal tracking challenges.
• The novel approach improves performance in nonlinear discrete-time systems.
• The actor-critic implementation validates the practical applicability of the proposed algorithm.