Cooperation in public goods games: Leveraging other-regarding reinforcement learning on hypergraphs

Cooperation is a self-organized collective behavior. It plays a significant role in the evolution of both ecosystems and human society. Reinforcement learning is different from imitation learning, offering a new perspective for exploring cooperation mechanisms. However, most existing studies with the public goods game (PGG) employ a self-regarding setup or are on pairwise interaction networks. Players in the real world, however, optimize their policies based not only on their histories but also on the histories of their coplayers, and the game is played in a group manner. In this work, we investigate the evolution of cooperation in the PGG under the other-regarding reinforcement learning evolutionary game on hypergraph by combining the Q-learning algorithm and evolutionary game framework, where other players' action history is incorporated and the game is played on hypergraphs. Our results show that as the synergy factor r[over ̂] increases, the parameter interval divides into three distinct regions-the absence of cooperation, medium cooperation, and high cooperation-accompanied by two abrupt transitions in the cooperation level near r[over ̂]_{1}^{*} and r[over ̂]_{2}^{*}, respectively. Interestingly, we identify regular and anticoordinated chessboard structures in the spatial pattern that positively contribute to the first cooperation transition but adversely affect the second. Furthermore, we provide a theoretical treatment for the first transition with an approximated r[over ̂]_{1}^{*} and reveal that players with a long-sighted perspective and low exploration rate are more likely to reciprocate kindness with each other, thus facilitating the emergence of cooperation. Our findings contribute to understanding the evolution of human cooperation, where other-regarding information and group interactions are commonplace.