Neural network-based finite horizon stochastic optimal control design for nonlinear networked control systems

Area of Science:

• Control Theory
• Artificial Intelligence
• Networked Systems

Background:

• Stochastic optimal control of nonlinear networked control systems (NNCSs) presents challenges due to terminal constraints, system uncertainties, and network imperfections like delays and packet losses.
• Traditional infinite horizon neuro-dynamic programming (NDP) schemes are inadequate for finite horizon NNCS problems with terminal constraints.

Purpose of the Study:

• To develop a novel time-based NDP scheme for solving the finite horizon optimal control problem in NNCSs.
• To address challenges posed by terminal constraints, system uncertainties, and network-induced delays and packet losses.

Main Methods:

• An online neural network (NN) identifier is employed to approximate the control coefficient matrix.
• Critic and actor NNs are utilized in conjunction with the identifier to determine a time-based stochastic optimal control input.
• Lyapunov theory is applied to demonstrate the uniform ultimate boundedness of closed-loop signals and NN weights.

Main Results:

• The proposed scheme effectively mitigates challenges associated with finite horizon optimal control in NNCSs.
• All closed-loop signals and NN weights are proven to be uniformly ultimately bounded.
• The approximated control input converges close to the optimal value within finite time.

Conclusions:

• The developed time-based NDP scheme offers an effective solution for finite horizon stochastic optimal control in NNCSs.
• The method ensures stability and convergence, demonstrating practical applicability through simulation results.