Combined control algorithm based on synchronous reinforcement learning for a self-balancing bicycle robot

In this paper, balance control of a bicycle robot is studied without either a trail or a mechanical regulator when the robot moves in an approximately rectilinear motion. Based on the principle of moment balance, an input nonaffine nonlinear dynamics model of the bicycle robot is established. A driving velocity condition is proposed to maintain the robot balance. The nonaffine nonlinear system is transformed into an affine nonlinear system by defining the equivalent control. Subsequently, a feedback linearization controller is designed for the equivalent control. We design a combined control algorithm of synchronous policy iteration based on the actor-critic architecture. The actor neural network (NN) is designed based on the feedback linearization control law. Weight tuning laws for the critic and actor NNs are proposed. The system closed-loop stability and convergence of the NN weights are guaranteed based on the Lyapunov analysis. The optimality of the equivalent control policy is guaranteed. To satisfy the driving velocity condition, the values of the steering angle and driving velocity are determined based on the optimal equivalent control. The effectiveness of the proposed algorithm is verified through simulations and real experiments.