Actor-critic-based optimal tracking for partially unknown nonlinear discrete-time systems.

Area of Science:

• Control Systems Engineering
• Robotics
• Artificial Intelligence

Background:

• Addressing the challenge of optimal control for nonlinear discrete-time systems with input constraints is crucial for many applications.
• Traditional methods often struggle to simultaneously handle feedback and feedforward control while respecting system limitations.

Purpose of the Study:

• To develop a partially model-free adaptive optimal control strategy for deterministic nonlinear discrete-time tracking problems.
• To incorporate input constraints directly into the optimal control design using a novel performance function.

Main Methods:

• Formulating an augmented system by combining tracking error and reference trajectory dynamics.
• Introducing a discounted performance function to derive both feedback and feedforward control components simultaneously.
• Deriving the discrete-time tracking Bellman and Hamilton-Jacobi-Bellman (HJB) equations.
• Employing an actor-critic reinforcement learning algorithm with two neural networks (actor and critic) for online policy learning.

Main Results:

• The proposed method successfully encodes input constraints into the optimization problem via a nonquadratic performance function.
• The actor-critic algorithm learns the optimal bounded control policy online without needing system drift dynamics.
• Simulation results demonstrate the effectiveness of the proposed adaptive optimal control solution.

Conclusions:

• The developed partially model-free adaptive optimal control approach effectively handles input constraints in nonlinear discrete-time systems.
• Simultaneous determination of feedback and feedforward control through a novel performance function offers a robust solution.
• The actor-critic reinforcement learning framework provides an efficient online learning mechanism for achieving optimal tracking control.