Network-Scale Traffic Signal Control via Multiagent Reinforcement Learning With Deep Spatiotemporal Attentive Network

Area of Science:

• Intelligent Transportation Systems
• Artificial Intelligence in Urban Planning
• Deep Reinforcement Learning Applications

Background:

• Intelligent traffic control systems are crucial for urban traffic planning and management.
• Deep reinforcement learning (RL) shows promise for improving intersection efficiency and reducing travel times.
• Existing RL algorithms often overlook the temporal and spatial characteristics of traffic intersections.

Purpose of the Study:

• To propose a novel multiagent reinforcement learning (MARL) algorithm, MARL-DSTAN, for traffic signal timing in large-scale road networks.
• To address the limitations of existing algorithms by incorporating spatiotemporal features of intersections.
• To enhance learning efficiency and accelerate algorithm convergence for practical traffic management.

Main Methods:

• Developed a deep spatiotemporal attentive neural network (MARL-DSTAN) model.
• Utilized graph convolutional networks (GCN) to capture spatial dependencies and attention mechanisms to weigh intersection importance.
• Integrated recurrent neural networks (RNN) for constrained exploration and employed a centralized training with decentralized execution approach.

Main Results:

• MARL-DSTAN effectively integrates spatial dependencies and intersection importance using GCN and attention.
• The inclusion of RNN in exploration enhances learning efficiency and sample value.
• Simulation results demonstrate superior performance compared to fixed timing schemes and baseline RL algorithms.

Conclusions:

• The proposed MARL-DSTAN algorithm offers a significant advancement in intelligent traffic signal control.
• By considering spatiotemporal characteristics, the model achieves greater efficiency in large-scale road networks.
• This approach holds potential for optimizing urban traffic flow and reducing congestion.