Using a single actor to output personalized policy for different intersections

Recent advances in Multi-Agent Reinforcement Learning (MARL) have demonstrated significant potential for adaptive traffic signal control. However, existing MARL approaches face dual challenges: Complete parameter sharing among agents leads to insufficient diversity in policy networks, while treating agents as heterogeneous entities (with non-shared parameters) better adapts to intersection heterogeneity but introduces training inefficiency and parameter explosion in large-scale road networks. Balancing agent performance with reduced computational resource consumption remains a critical challenge that demands urgent resolution. To address these issues, we propose the Hyper-Action Multi-Head Proximal Policy Optimization (HAMH-PPO) method, which enhances personalized representation capabilities through value functions. This approach constructs K shared value function libraries to provide differentiated value estimation for traffic networks, while employing a hyper-network to dynamically generate adaptive weights for these libraries across different intersections. Consequently, a single shared network can learn diverse control strategies that accommodate varying observations. The experimental results showed that HAMH-PPO outperforms traditional traffic signal methods by 39.44% and achieves a 29.8% improvement over the suboptimal solution in complex, large-scale synthetic road networks. Crucially, HAMH-PPO maintains the benefits of parameter sharing while optimizing algorithmic performance and reducing computational costs.