Federated deep reinforcement learning-based urban traffic signal optimal control

Area of Science:

• Artificial Intelligence
• Transportation Engineering
• Distributed Systems

Background:

• Deep reinforcement learning (RL) faces challenges in cross-domain traffic signal control, including slow learning and poor generalization.
• Existing methods struggle with non-independent and heterogeneous environmental data in real-world traffic scenarios.

Purpose of the Study:

• To develop a cross-domain intelligent traffic signal control method using federated Proximal-Policy Optimization (PPO).
• To address slow learning speeds and enhance model generalization for traffic signal optimization.
• To ensure information security and data privacy during distributed joint training.

Main Methods:

• Proposed a federated PPO approach for distributed joint training of agents across different traffic domains.
• Designed state, action, and reward functions, and optimized federated collaboration parameters.
• Implemented a novel state interaction method and reward function to improve agent collaboration and communication efficiency.

Main Results:

• Significantly reduced average vehicle waiting time by up to 27.34% compared to fixed timing.
• Achieved convergence speeds up to 47.69% faster than individual PPO and 45.35% faster than aggregated PPO.
• Demonstrated excellent robustness across various traffic flow settings.

Conclusions:

• The federated PPO method effectively optimizes intersection access efficiency and traffic flow.
• The approach enhances learning efficiency, fast convergence, and model generalization in cross-domain traffic control.
• Ensures information security and data privacy while improving communication efficiency in federated learning for traffic systems.