Koopman-Driven Linearized Model-Based Offline Planning With Application to Freeway Ramp Metering.

Area of Science:

• Control Systems Engineering
• Machine Learning
• Transportation Science

Background:

• Freeway ramp metering (RM) is crucial for traffic flow optimization.
• Existing methods struggle with the nonlinear dynamics of traffic control.
• Model predictive control (MPC) and reinforcement learning (RL) offer potential but face challenges in complex systems.

Purpose of the Study:

• To propose a novel model-based planning framework, Koopman-driven linearized model-based offline planning (KLMOP), for freeway ramp metering.
• To integrate MPC and offline RL using Koopman operator theory for a linear Markov decision process (MDP).
• To develop a computationally efficient and adaptable approach for linearizing nonlinear control problems.

Main Methods:

• Utilized Koopman operator theory to learn and model system dynamics, reward, and value functions in a latent space via a Koopman-based latent dynamical model (KLDM).
• Employed a pessimistic value iteration (PEVI) algorithm for policy optimization within the linear MDP framework.
• Applied contrastive learning to ensure the quality of the latent representation for accurate reward prediction and efficient policy development.
• Integrated these components into an MPC-based planning policy to solve linear MPC problems in the latent space.

Main Results:

• KLMOP demonstrated significant improvements in computational efficiency compared to existing baseline methods.
• The framework achieved superior control performance in extensive simulation studies for ramp metering.
• The approach successfully linearized complex nonlinear control problems through learning-based methods.

Conclusions:

• KLMOP offers a theoretically grounded and computationally efficient solution for freeway ramp metering.
• The learning-based design enhances adaptability for broader applications beyond RM.
• This framework advances the integration of advanced control and machine learning techniques in intelligent transportation systems.