Neural network approach to continuous-time direct adaptive optimal control for partially unknown nonlinear systems

Area of Science:

• Control Theory
• Machine Learning
• Nonlinear Systems

Background:

• Optimal control for nonlinear systems is challenging due to unknown dynamics.
• Infinite horizon cost functions require robust control strategies.
• Existing methods often require system model knowledge or lack online adaptation.

Purpose of the Study:

• To develop an online direct adaptive optimal control algorithm for nonlinear systems with infinite horizon cost.
• To ensure closed-loop dynamic stability without prior knowledge of system dynamics.
• To introduce a novel hybrid control structure combining continuous-time control and adaptive supervision.

Main Methods:

• Utilizes a reinforcement learning scheme, specifically Policy Iterations.
• Employs an Actor/Critic neural network structure to represent control policy and performance.
• Trains neural networks for optimal controller and cost function approximation.
• Proves convergence under realistic assumptions of imperfect neural network representations.

Main Results:

• The algorithm converges online to the optimal control solution.
• Closed-loop dynamic stability is guaranteed.
• A novel hybrid control structure is presented, distinct from standard controllers.
• Simulations on second-order nonlinear systems demonstrate effectiveness.

Conclusions:

• The proposed online adaptive optimal control method is effective for nonlinear systems.
• The hybrid control structure offers a new paradigm for adaptive control.
• The approach guarantees stability and convergence even with imperfect function approximation.