Stochastic optimal controller design for uncertain nonlinear networked control system via neuro dynamic programming.

Area of Science:

• Control Engineering
• Artificial Intelligence
• Systems Science

Background:

• Nonlinear networked control systems (NNCS) face challenges from system nonlinearities and network imperfections like delays and packet losses.
• Traditional neuro dynamic programming (NDP) methods using value/policy iterations are unsuitable for real-time control.
• Output feedback controllers are preferred for practical implementation in NNCS.

Purpose of the Study:

• To develop a novel NNCS representation for output feedback control, incorporating system uncertainties and network imperfections.
• To design a time-based stochastic optimal controller for NNCS using an online neural network (NN) identifier and critic-action NNs.
• To achieve real-time control without relying on value or policy iterations.

Main Methods:

• Introduced a new NNCS representation using input/output measurements to handle uncertainties and network imperfections.
• Employed an online NN identifier to estimate the control coefficient matrix for controller design.
• Utilized critic and action NNs with the NN identifier for forward-in-time, time-based stochastic optimal control, updating value functions and control inputs at each sampling instant.

Main Results:

• Demonstrated uniform ultimate boundedness of closed-loop signals and NN weights using Lyapunov theory.
• Showed convergence of the approximated control input towards its target value over time.
• Validated the effectiveness of the proposed control scheme through simulation results.

Conclusions:

• The proposed time-based NDP approach effectively addresses stochastic optimal control for NNCS with uncertainties and network imperfections.
• The method enables real-time control by avoiding traditional iterative techniques and utilizing output feedback.
• The developed NN weight update laws ensure system stability and control accuracy.