Neural-network-based discounted optimal control via an integrated value iteration with accuracy guarantee.

Area of Science:

• Control Theory
• Machine Learning
• Nonlinear Systems

Background:

• Optimal control problems often involve unknown nonlinear system dynamics.
• Accurate system identification is crucial for effective control.
• Traditional methods may struggle with complex nonlinearities.

Purpose of the Study:

• To develop a data-based value iteration algorithm for discounted optimal control.
• To improve the precision of nonlinear system dynamics identification.
• To guarantee the approximation accuracy of the optimal value function.

Main Methods:

• Establishing a model neural network with biases for system identification.
• Training the neural network using gradient descent algorithm for weight and bias updates.
• Analyzing uniform ultimate boundedness stability using the Lyapunov approach.
• Integrating value iteration with discounted cost for optimal value function approximation.

Main Results:

• The proposed algorithm effectively identifies unknown nonlinear system dynamics.
• The neural network with biases demonstrates improved identification precision.
• The Lyapunov analysis confirms uniform ultimate boundedness stability.
• The integrated value iteration guarantees approximation accuracy of the optimal value function.

Conclusions:

• The developed data-based value iteration algorithm is effective for discounted optimal control.
• The inclusion of biases in the model neural network enhances identification accuracy.
• The algorithm's stability and approximation accuracy are theoretically analyzed and practically demonstrated.