An iterative Q-learning based global consensus of discrete-time saturated multi-agent systems

This paper addresses the consensus problem of discrete-time multiagent systems (DTMASs), which are subject to input saturation and lack of the information of agent dynamics. In the previous works, the DTMASs with input saturation can achieve semiglobal consensus by utilizing the low gain feedback (LGF) method, but computing the LGF matrices by solving the modified algebraic Riccati equation requires the knowledge of agent dynamics. In this paper, motivated by the reinforcement learning method, we propose a model-free Q-learning algorithm to obtain the LGF matrices for the DTMASs achieving global consensus. Firstly, we define a Q-learning function and deduce a Q-learning Bellman equation, whose solution can work out the LGF matrix. Then, we develop an iterative Q-learning algorithm to obtain the LGF matrix without the requirement of the knowledge about agent dynamics. Moreover, the DTMASs can achieve global consensus. Lastly, some simulation results are proposed to validate the effectiveness of the Q-learning algorithm and show the effect on the rate of convergence from the initial states of agents and the input saturation limit.