Adaptive formation control of leader-follower mobile robots using reinforcement learning and the Fourier series...

In this paper, a formation controller for leader-follower mobile robots is presented based on reinforcement learning and the Fourier series expansion. The controller is designed based on the dynamical model in which permanent magnet direct-current (DC) motors are included as actuator. Thus, motor voltages are the control signals and are designed based on the actor-critic strategy which is a well-known approach in the field of reinforcement learning. Stability analysis of formation control of leader-follower mobile robots using the proposed controller verifies that the closed-loop system is globally asymptotically stable. Due to the existence of sinusoidal terms in the model of mobile robots, the Fourier series expansion has been selected to construct the actor and critic, while previous related works utilized neural networks in actor and critic. In comparison with neural networks, the Fourier series expansion is simpler and involves the designer in fewer tuning parameters. In simulation studies, it has been assumed that some follower robots can play the role of leader for the other follower robots behind it. Simulation results show that there is no need to use large number of the sinusoidal terms in the Fourier series expansion and just the first three terms can overcome uncertainties. In addition, the proposed controller reduced the performance index of tracking errors considerably in comparison with radial basis function neural networks (RBFNN).