Using a single actor to output personalized policy for different intersections.

Area of Science:

• Artificial Intelligence
• Transportation Engineering
• Reinforcement Learning

Background:

• Multi-Agent Reinforcement Learning (MARL) shows promise for adaptive traffic signal control.
• Existing MARL methods struggle with policy diversity and training efficiency in large-scale networks.
• Balancing agent performance with computational cost is a key challenge.

Purpose of the Study:

• To develop an efficient MARL approach for adaptive traffic signal control.
• To enhance policy diversity and reduce computational complexity in large-scale road networks.
• To improve traffic flow and reduce delays through intelligent signal management.

Main Methods:

• Proposed the Hyper-Action Multi-Head Proximal Policy Optimization (HAMH-PPO) method.
• Utilized K shared value function libraries for differentiated value estimation.
• Employed a hyper-network to generate adaptive weights for value libraries across intersections.
• Enhanced personalized representation capabilities through value functions.

Main Results:

• HAMH-PPO outperformed traditional traffic signal methods by 39.44%.
• Achieved a 29.8% improvement over suboptimal solutions in complex, large-scale synthetic road networks.
• Demonstrated effective learning of diverse control strategies for varying traffic observations.
• Maintained benefits of parameter sharing while optimizing performance and reducing costs.

Conclusions:

• HAMH-PPO offers an effective solution for adaptive traffic signal control in large-scale networks.
• The method balances agent performance with reduced computational resource consumption.
• This approach enhances policy diversity and adaptability in MARL for traffic management.