Traffic Signal Control Using Hybrid Action Space Deep Reinforcement Learning.

Area of Science:

• Artificial Intelligence
• Transportation Engineering
• Computer Science

Background:

• Intelligent traffic signal control (TSC) increasingly utilizes Deep Reinforcement Learning (DRL).
• Existing DRL frameworks for TSC are limited to either discrete (phase selection) or continuous (phase duration) control.
• A flexible framework integrating both discrete and continuous DRL approaches for TSC is lacking.

Purpose of the Study:

• To propose a novel hybrid Deep Reinforcement Learning (DRL) framework for traffic signal control (TSC).
• To develop an approach capable of simultaneously deciding the appropriate traffic light phase and its duration.
• To address the limitations of existing discrete or continuous DRL-based TSC systems.

Main Methods:

• Adapted a hybrid Deep Reinforcement Learning (DRL) approach, specifically Parameterized Deep Q-Networks (P-DQN).
• Implemented a hierarchical decision-making process within the P-DQN architecture.
• Evaluated the proposed framework using the Simulation of Urban MObility (SUMO) traffic simulator.

Main Results:

• The hybrid DRL framework demonstrated superior performance compared to existing benchmarks.
• Achieved a 22.20% reduction in average vehicle queue length.
• Reduced average vehicle travel time by 5.78%.

Conclusions:

• The proposed hybrid DRL approach offers a flexible and effective solution for traffic signal control (TSC).
• Simultaneously optimizing traffic light phase and duration leads to significant improvements in traffic efficiency.
• The P-DQN customization enables hierarchical decision-making for enhanced TSC performance.