Policy-Iteration-Based Finite-Horizon Approximate Dynamic Programming for Continuous-Time Nonlinear Optimal Control.

Area of Science:

• Optimal Control Theory
• Dynamic Programming
• Machine Learning

Background:

• The Hamilton-Jacobi-Bellman (HJB) equation is crucial for continuous-time optimal control problems (OCPs).
• Solving finite-horizon (FH) HJB equations is challenging due to the unknown partial time derivative of the value function.

Purpose of the Study:

• To develop a novel approach for solving continuous-time finite-horizon optimal control problems (CT FH OCPs).
• To address the challenge posed by the partial time derivative in FH HJB equations.

Main Methods:

• A novel link is established between the partial time derivative and the terminal-time utility function.
• A finite-horizon approximate dynamic programming (ADP) algorithm is proposed using an actor-critic framework.
• Multilayer neural networks (NNs) are integrated into the actor-critic architecture for enhanced capability.

Main Results:

• The proposed policy iteration (PI) technique effectively solves CT FH OCPs.
• The FH ADP algorithm demonstrates convergence and optimality properties.
• The NN-based actor-critic approach is applicable to general nonlinear and complex systems.

Conclusions:

• The study presents a significant advancement in solving CT FH OCPs, particularly for complex systems.
• The developed algorithm offers a robust and effective solution for optimal control challenges.
• Simulations on LQR and vehicle tracking problems validate the algorithm's effectiveness.