An expert-guided multi-agent reinforcement learning framework with balanced exploration for uncontrolled...

Balancing safety and efficiency in autonomous driving at an uncontrolled intersection poses a fundamental challenge, for which MARL offers a powerful solution. However, current multi-agent reinforcement learning (MARL) algorithms typically encounter limitations, such as low training efficiency and convergence to suboptimal strategies. To overcome these challenges, this study proposes a novel value-based MARL framework incorporating three primary innovations: 1) a dynamic value-learning objective function integrating human decision-making elements, enabling agents to dynamically adjust their reliance on expert guidance based on the actual rewards of actions, thus mitigating inefficiencies from random exploration; 2) a nonlinear incentive regularization term designed to promote optimal joint actions and suppress suboptimal behaviors, significantly enhancing the model's asymptotic reward performance; and 3) an ego external attention mechanism developed to improve agents' perception of ego and surrounding vehicles, thus bolstering decision-making stability during frequent interactions with complex environments.Experiments conducted in unsignalized intersection scenarios encompassing three representative traffic conflict intensities demonstrate that the proposed algorithm significantly improves decision-making performance in complex multi-vehicle interactions. In the four-vehicle high-conflict scenario, it achieves approximately 211% higher training efficiency compared to the VDN baseline, superior safety rates over game-theoretic methods, and markedly higher asymptotic rewards than all compared baselines, demonstrating its efficacy.