Multiple Environment Integral Reinforcement Learning-Based Fault-Tolerant Control for Affine Nonlinear Systems

This paper studies the fault-tolerant control (FTC) problem for unknown affine nonlinear systems with actuators faults. The considered types of faults are stuck (lock-in-place), loss-in-effectiveness (LIE), and bias, under which a part of the actuators is disabled. The objective is to find the remaining (not fully LIE) actuators, and manipulate them to obtain the best achievable performance in real time. First, considering that the best achievable performance is determined by the remaining actuators, a set of basic policies is predesigned with multiple levels of performances for different groups of activated actuators. Second, an identifier is designed based on history data to find the remaining actuators and, thus, the suitable predesigned basic policy. Third, to further accommodate the partial LIEs and biases, a compensator works together with the selected basic policy, to build the predesigned performance. In addressing the FTC problem, several techniques are developed: adjustable mechanisms are novelly integrated to deal with the state-dependent nonlinearities in neural network (NN) approximation, disturbances, and mismatch errors; history data are newly applied to estimate the faulty parameters; and a compensator is specially designed to deal with LIEs and biases in different input channels. Also in theory, the convergences of algorithms and the stability of closed-loop systems are proved, by formally giving the invariant sets of the initial state and the NN weights. Unlike the existing FTC methods dealing with LIE and bias based on model information to optimize the tracking error, this result can handle stuck faults without knowing system dynamics and satisfy different levels of performances described by Hamilton-Jacobi-Bellman equations. Finally, a simulation example of quadrotor unmanned aerial vehicle is given to verify the effectiveness of the proposed FTC scheme.