Transfer learning-based safe Q-learning for discrete-time nonlinear systems with asymmetric state constraints

This article investigates the problem of safe control for discrete-time (DT) nonlinear systems subject to asymmetric state constraints. A novel transfer learning-based safe Q-learning (TL-SQL) algorithm is developed within the adaptive dynamic programming (ADP) framework. To support the proposed TL-SQL algorithm, a novel revived transformation formulation for DT systems is derived, serving as the basis for transferring control policies from unconstrained source systems to constrained target systems. Building upon this foundation, the proposed TL-SQL algorithm improves policy transfer under asymmetric constraints in three aspects. Firstly, unlike traditional state transformation methods, the revived transformation alleviates the adverse impact of state transformation on the optimality of the original system. Secondly, compared with control-barrier-function-based approaches, it avoids complicating the utility function structure. Thirdly, by decoupling policy training from constraint handling, it enables efficient adaptation to changing constraints without retraining. Theoretical analysis ensures stability and safety of the transferred policy. Simulation results validate the effectiveness of TL-SQL under asymmetric constraint scenarios.