A survey of adaptive optimal control theory

Area of Science:

• Control Engineering
• Artificial Intelligence
• Machine Learning

Background:

• Optimal control is crucial for system performance.
• Dynamic programming (DP) and reinforcement learning (RL) are foundational optimization techniques.
• Adaptive dynamic programming (ADP) integrates DP with learning for complex control problems.

Purpose of the Study:

• To survey recent advancements in optimal control based on ADP.
• To classify and summarize research in optimization methods, constraints, algorithm design, and engineering applications.
• To identify future research directions in ADP-based optimal control.

Main Methods:

• Review of DP and RL algorithms.
• Classification and summarization of optimal control research.
• Analysis of critic-actor neural network structures in ADP.

Main Results:

• ADP, particularly with critic-actor neural networks, shows significant promise for optimal control.
• Developed optimal control algorithms have been successfully applied in practical engineering.
• The survey provides a comprehensive overview of the field's progress and challenges.

Conclusions:

• Optimal control via ADP is a rapidly developing field with broad application prospects.
• Further research is needed to explore new algorithms and address complex engineering challenges.
• ADP-based optimal control is transitioning from theoretical research to practical implementation.