Cooperative or competitive? Resolving social dilemmas in autonomous vehicles through evolutionary game theory

With the large-scale deployment of autonomous vehicles (AVs), AV-human-driven vehicle (HV) interactions are increasingly common. AVs face a social dilemma: competitive behavior raises ethical and public acceptance concerns, whereas cooperative behavior can invite exploitation and degrade efficiency. We use Evolutionary Game Theory (EGT) to model long-run adaptation between AVs and HVs and quantify agent sociality via a data-calibrated Social Value Orientation (SVO) metric. After calibrating HV social preferences from unprotected left-turn trajectories, we incorporate HV heterogeneity into a two-population EGT with cooperative and competitive types. SVO-informed rewards are used to construct payoff matrices for replicator analyses to identify evolutionarily stable strategies (ESS). Experiments show that AV policies with moderate egoism mitigate the social dilemma and tend to achieve population-level dominance in both roles (left-turning and straight-going), whereas overly cooperative policies are evolutionarily unstable. Moreover, AVs benefit from opponent-aware, dynamically adjustable sociality to accommodate diverse HV preferences. To test the theory, we run agent-based imitation simulations. Sensitivity analyses indicate that AV advantages are hard to observe at low market penetration but become pronounced as penetration approaches about 50%, after which convergence accelerates. Overall, the framework clarifies when and why AV sociality preferences succeed over time, offering actionable guidance for designing adaptive, socially compatible AV decision policies in mixed traffic.