Beam management optimization for V2V communications based on deep reinforcement learning

Intelligent connected vehicles have garnered significant attention from both academia and industry in recent years as they form the backbone of intelligent transportation and smart cities. Vehicular networks now exchange a range of mixed information types, including safety, sensing, and multimedia, due to advancements in communication and vehicle technology. Accordingly, performance requirements have also evolved, prioritizing higher spectral efficiencies while maintaining low latency and high communication reliability. To address the trade-off between communication spectral efficiency, delay, and reliability, the 3rd Generation Partnership Project (3GPP) recommends the 5G NR FR2 frequency band (24 GHz to 71 GHz) for vehicle-to-everything communications (V2X) in the Release 17 standard. However, wireless transmissions at such high frequencies pose challenges such as high path loss, signal processing complexity, long pre-access phase, unstable network structure, and fluctuating channel conditions. To overcome these issues, this paper proposes a deep reinforcement learning (DRL)-assisted intelligent beam management method for vehicle-to-vehicle (V2V) communication. By utilizing DRL, the optimal control of beam management (i.e., beam alignment and tracking) is achieved, enabling a trade-off among spectral efficiency, delay, and reliability in complex and fluctuating communication scenarios at the 5G NR FR2 band. Simulation results demonstrate the superiority of our method over the 5G standard-based beam management method in communication delay, and the extended Kalman Filter (EKF)-based beam management method in reliability and spectral efficiency.