Two-Stage Homotopic Learning for Data-Driven Cluster Consensus in Multiagent Systems

In this article, a novel homotopic reinforcement learning (RL) framework is proposed to address the distributed cluster consensus problem in continuous-time multiagent systems (MASs). For the first time, the investigated issue is formulated as a zero-sum differential game using a proposed minmax game policy, in which a local regulation error is introduced to characterize the deviation of each agent from its assigned leader. Based on this formulation, a set of group game algebraic Riccati equations is derived to obtain the optimal control law. To overcome reliance on known system models, these equations are solved using a data-driven homotopic policy-iteration scheme that leverages online state and input information. In contrast to conventional learning schemes, the proposed approach embeds a homotopic process that relaxes the requirement for an admissible initial policy. Rigorous stability and convergence analyses are provided, and the effectiveness of the proposed method is further demonstrated through theoretical analysis and numerical simulations.