Cooperative Deep Reinforcement Learning for Large-Scale Traffic Grid Signal Control

Area of Science:

• Intelligent Transportation Systems
• Artificial Intelligence
• Traffic Engineering

Background:

• Traffic congestion is a major challenge in urban areas, exacerbated by complex traffic dynamics.
• Traditional traffic control methods struggle with large-scale systems and dynamic conditions.
• Reinforcement learning (RL) offers potential but faces scalability issues with numerous signal lights and intersections.

Purpose of the Study:

• To propose a novel cooperative deep reinforcement learning (Coder) framework for efficient traffic congestion reduction.
• To address the challenge of large action spaces in RL for traffic signal control on extensive road networks.
• To enhance the ability of RL agents to monitor multiple signal lights and manage complex traffic dynamics.

Main Methods:

• Developed a hierarchical framework (Coder) with multiple regional agents and a centralized global agent.
• Regional agents learn policies for localized areas with limited actions.
• A global agent aggregates regional successes to form a Q-function for the entire traffic grid.

Main Results:

• The Coder framework demonstrated significant reductions in traffic congestion.
• Simulations showed an average reduction of 30% in waiting vehicles during high-density traffic flows.
• The hierarchical approach effectively managed the exponential growth of the action space in large-scale traffic grids.

Conclusions:

• The proposed Coder framework offers a scalable and effective solution for traffic congestion reduction using deep reinforcement learning.
• Hierarchical decomposition is a viable strategy for tackling complex RL problems in intelligent transportation.
• Coder significantly improves traffic flow efficiency in simulated high-density conditions.