Lightweight Reinforcement Learning for Priority-Aware Spectrum Management in Vehicular IoT Networks | JoVE Visualize

Area of Science:

Computer Science
Electrical Engineering
Artificial Intelligence

Background:

Vehicular Internet of Things (V-IoT) is crucial for intelligent transportation systems (ITSs), supporting diverse applications with strict Quality-of-Service (QoS) demands.
Existing spectrum management techniques struggle with the dynamic nature of V-IoT networks and the computational demands of advanced solutions like deep reinforcement learning (DRL).
There is a need for efficient, real-time spectrum management frameworks suitable for roadside unit (RSU) deployment in V-IoT.

Purpose of the Study:

To propose a lightweight and interpretable reinforcement learning (RL)-based spectrum management framework for V-IoT networks.
To introduce and evaluate two enhanced Q-Learning variants: VPADQ-C and Q-UCB, for improved spectrum allocation.
To provide a baseline (Risk-Aware Heuristic) for comparing learning-driven approaches against traditional methods.

Main Methods:

Developed Value-Prioritized Action Double Q-Learning with Constraints (VPADQ-C) using Constrained Markov Decision Process (CMDP) and online primal-dual optimization.
Introduced contextual Q-Learning with Upper Confidence Bound (Q-UCB) incorporating uncertainty-aware exploration and Success-Rate Prior (SRP).
Created a comprehensive simulation framework modeling traffic, fading, and energy dynamics to assess performance metrics.

Main Results:

VPADQ-C demonstrated superior energy efficiency (≈8.425×10^7 bits/J) and over 60% reduction in interruption probability.
Q-UCB achieved faster convergence (≈190 episodes), lowest blocking probability (≈0.0135), and minimal mean delay (≈0.351 ms).
Both methods outperformed conventional Q-Learning and Double Q-Learning, maintaining fairness (≈0.364) and throughput (≈28 Mbps) with scalable training times.

Conclusions:

The proposed RL-based framework offers a viable solution for real-time spectrum management in V-IoT.
VPADQ-C and Q-UCB provide distinct advantages in energy efficiency, reliability, convergence speed, and latency.
The frameworks are suitable for large-scale V-IoT deployments, meeting URLLC-grade requirements under dense vehicular traffic.