A Cascaded Multi-Agent Reinforcement Learning-Based Resource Allocation for Cellular-V2X Vehicular Platooning...

Area of Science:

• Intelligent Transportation Systems (ITS)
• Autonomous Driving
• Wireless Communications

Background:

• Vehicular platooning enhances roadway utilization and fuel efficiency but traditionally relies on limited onboard computation and Dedicated Short-Range Communication (DSRC).
• The integration of 5G and Multi-access Edge Computing (MEC) offers opportunities to offload computations for advanced platooning systems.
• Key challenges include managing latency-sensitive radio resources and minimizing the Age of Information (AoI) for safety-critical Cooperative Awareness Messages (CAM).

Purpose of the Study:

• To investigate latency-sensitive radio resource management and AoI in 5G-enabled vehicular platooning systems.
• To address the need for a more sophisticated framework than Multi-Agent Deep Deterministic Policy Gradient (MADDPG) for handling multiple, correlated objectives in vehicular networks.
• To propose and evaluate a novel Cascaded MADDPG (CMADDPG) framework for improved performance in decentralized resource allocation.

Main Methods:

• Development of a novel Cascaded MADDPG (CMADDPG) framework designed to train cascaded target critics for collaborative agent behavior.
• Implementation of techniques to circumvent the estimation bias phenomenon, a common limitation in MADDPG.
• Experimental analysis to evaluate the convergence, reliability of CAM message dissemination, and AoI performance of the proposed CMADDPG algorithm.

Main Results:

• The CMADDPG framework demonstrates rapid convergence with minimal distortions.
• Cooperative Awareness Messages (CAM) are disseminated reliably with 99% probability, ensuring timely safety information.
• The average Age of Information (AoI) is maintained within 5-10 ms, guaranteeing high Quality of Service (QoS) even with channel uncertainties and increasing platoon sizes.

Conclusions:

• The proposed CMADDPG algorithm effectively addresses multiple objectives in vehicular platooning, outperforming conventional approaches.
• The framework exhibits robustness in decentralized resource allocation under channel uncertainties and varying mobility conditions.
• CMADDPG proves resilient to increasing platoon sizes, offering a scalable solution for future autonomous driving systems.