IHG-MA: Inductive heterogeneous graph multi-agent reinforcement learning for multi-intersection traffic signal...

Area of Science:

• Artificial Intelligence
• Traffic Engineering
• Computer Science

Background:

• Multi-agent deep reinforcement learning (MDRL) is used for traffic signal control but faces challenges with policy transfer, adaptability to varying vehicle numbers, and capturing heterogeneous traffic network features.
• Current MDRL methods often use specialized settings that limit generalization to new traffic networks.
• Existing deep learning and homogeneous graph neural network approaches struggle with the complexity and dynamic nature of traffic environments.

Purpose of the Study:

• To propose a novel algorithm, Inductive Heterogeneous Graph Multi-agent Actor-critic (IHG-MA), for enhanced multi-intersection traffic signal control.
• To address the limitations of current MDRL algorithms in policy transfer, generalization, and handling heterogeneous traffic data.
• To develop a more flexible and adaptable traffic signal control system.

Main Methods:

• Developed an Inductive Heterogeneous Graph (IHG) neural network for representation learning, capable of encoding heterogeneous node and graph features.
• Implemented a decentralized cooperative Multi-Agent Actor-Critic (MA) framework for policy learning using learned embeddings.
• Utilized Q-value and policy loss for optimizing the IHG-MA algorithm.

Main Results:

• The IHG-MA algorithm demonstrated superior performance compared to state-of-the-art methods across various traffic metrics.
• The inductive nature of IHG enabled effective representation learning for unseen nodes and new traffic network structures.
• The algorithm successfully handled heterogeneous features and structural information within traffic networks.

Conclusions:

• The IHG-MA algorithm represents a promising advancement in multi-intersection traffic signal control.
• The proposed approach enhances policy transferability and adaptability to diverse and dynamic traffic conditions.
• This work offers a robust solution for optimizing traffic flow by effectively leveraging heterogeneous graph information.