Enhancing multi-scenario applicability of freeway variable speed limit control strategies using continual learning.

Area of Science:

• Intelligent Transportation Systems
• Machine Learning
• Control Theory

Background:

• Variable Speed Limit (VSL) systems enhance freeway operations by dynamically adjusting speed limits for scenarios like traffic jams and crash prevention.
• Deep Reinforcement Learning (DRL) is used to develop VSL strategies by mapping traffic conditions to speed limits.
• DRL faces performance decay in multi-scenario applications due to 'scenario forgetting,' where previously learned information is lost during new scenario training.

Purpose of the Study:

• To introduce a continual learning approach to improve the multi-scenario applicability of VSL control strategies.
• To address the 'scenario forgetting' problem in DRL-based VSL systems.
• To enhance the robustness and generalization of VSL control strategies across diverse traffic conditions.

Main Methods:

• A gradient projection memory (GPM) based neural network parameter updating method was proposed.
• This method constrains gradient updates to preserve learned memories from previous scenarios during new scenario training.
• The approach was evaluated using three freeway operation scenarios simulated in SUMO.

Main Results:

• The continual learning approach reduced performance decay in previously trained scenarios by 17.76%.
• The multi-scenario VSL control strategies decreased speed standard deviation by 28.77% and average travel time by 7.25%.
• The generalization capabilities of the continual learning-based VSL approach were assessed.

Conclusions:

• Continual learning effectively mitigates scenario forgetting in DRL-based VSL systems.
• The proposed GPM method enhances the long-term performance and applicability of VSL strategies in dynamic, multi-scenario environments.
• The developed VSL strategies offer significant improvements in traffic flow efficiency and safety.