Proximal Policy Optimization-guided novelty-aware multi-strategy particle swarm optimization for adaptive traffic signal optimization

Traffic signal timing is a non-linear, stochastic and time-varying optimization problem. Particle Swarm Optimization (PSO), Genetic Algorithm (GA) and Ant Colony Optimization (ACO) run one fixed search strategy for the whole run, which often leads to early convergence on a poor timing plan. This study proposes PPO-NAMSPSO, which changes the search strategy during the run.

A Proximal Policy Optimization (PPO) agent observes the state of a Novelty-Aware Multi-Strategy PSO (NAMSPSO) and, at each iteration, selects one of six operators: standard PSO update only, novelty-guided exploration, elite-guided update, mutation, restart, or local search. Candidates are ranked on waiting time, queue length and behavioral novelty. An elite pool guides the swarm, and stagnation recovery reinitializes the worst half of the particles after four iterations without improvement. We tested the framework in the Simulation of Urban Mobility (SUMO) on a synthetic 3 × 3 grid and on an OpenStreetMap-derived network with 25 signalized intersections, at medium and high demand. PSO, GA, ACO and fixed-time control were compared, and each optimizer was tuned with Optuna under the same budget. Each optimizer was trained with five seeds and evaluated over 100 runs.

Against PSO, average waiting time fell from 105.52 s to 96.97 s (8.10%) on the grid and from 48.88 s to 33.07 s (32.35%) on the OSM network, where its standard deviation fell from 17.42 s to 3.54 s. Under high demand on the grid, waiting time was 207.72 s against 315.91 s for PSO. PPO-NAMSPSO had the lowest waiting time of the five methods in all three reported comparisons. On the OSM network, ANOVA and Kruskal-Wallis tests found differences among methods (p < 0.001), and Holm-corrected Welch tests against each method gave p ≤ 1.17 × 10^-5, with Cohen's d from 0.71 to 2.33. Removing the PPO agent raised mean waiting time from 60.38 s to 64.91 s under high demand.

The gain was larger on the OSM network than on the grid. The agent used all six operators, and the elite-guided update was chosen most often (20.61% of selections). Demand was synthetic, so validation on field data remains open.