Error bounds of adaptive dynamic programming algorithms for solving undiscounted optimal control problems.

Area of Science:

• Control Theory
• Optimization
• Machine Learning

Background:

• Optimal control problems are crucial in engineering and economics.
• Adaptive dynamic programming (ADP) offers a powerful framework for solving these problems.
• Existing ADP methods often rely on contraction assumptions, limiting their applicability.

Purpose of the Study:

• To establish error bounds for ADP algorithms in undiscounted infinite-horizon optimal control.
• To address approximation errors in value function and control policy updates.
• To introduce a new assumption to overcome limitations of contraction assumptions.

Main Methods:

• Developing novel error conditions for approximate value iteration.
• Establishing error bounds for approximate policy iteration and approximate optimistic policy iteration.
• Utilizing critic and action neural networks for function and policy approximation.

Main Results:

• Derived error bounds for adaptive dynamic programming algorithms.
• Demonstrated that approximate value functions can converge to a finite neighborhood of the optimal value function.
• Validated the effectiveness of the developed algorithms through a simulation example.

Conclusions:

• The proposed ADP algorithms provide robust error bounds for nonlinear systems.
• The use of neural networks enables practical implementation of these advanced control strategies.
• This work advances the theoretical understanding and practical application of ADP in optimal control.