Optimal Adaptive Critic Learning Control of Continuous-Time Nonlinear Systems: A Differential Riccati Equation-Based Method

This brief introduces an optimal adaptive critic learning control for continuous-time (CT) nonlinear systems. First, a single-critic neural network (NN) is employed to approximate the solution of the derived Hamilton-Jacobi-Bellman (HJB) equation, with a filtered Bellman residual serving as the mechanism for updating the critic NN weights. To eliminate the necessity of manual tuning of fixed adaptation parameters, the weight error dynamics are reformulated as a CT optimal control problem. Consequently, solving the associated differential Riccati equation (DRE) in real time enables the derivation of an optimal adaptive law that effectively balances the convergence performance and system robustness. Under the condition that regressor signals satisfy a persistent excitation (PE) condition, the critic NN weight errors exponentially converge to a bounded region determined by the approximation residual. The proposed algorithm is validated through experiments on an axial piston pump. The results demonstrate that the proposed learning approach enables fast critic weight convergence while reducing the weight update fluctuations.