Two-Stage Multiarmed Bandit for Reconfigurable Intelligent Surface Aided Millimeter Wave Communications

Area of Science:

• Wireless Communications
• Signal Processing
• Machine Learning

Background:

• Reconfigurable intelligent surfaces (RIS) enhance millimeter wave (mmWave) communication range.
• Optimal phase shifts (PSs) for mmWave transmitters (TX) and RIS are crucial for effective coverage.
• Estimating mmWave channel state information (CSI) for TX and RIS is challenging.

Purpose of the Study:

• To propose a codebook-based phase shifter approach for mmWave TX and RIS.
• To develop an online learning strategy for optimizing TX and RIS phase shifts.
• To address the challenge of estimating mmWave CSI by leveraging multi-armed bandit (MAB) games.

Main Methods:

• A nested two-stage stochastic MAB strategy is proposed for phase shift optimization.
• The strategy involves alternating optimization between mmWave TX and RIS phase shift vectors.
• Thompson sampling (TS) and Upper Confidence Bound (UCB) algorithms are implemented within the MAB framework.

Main Results:

• Simulation results demonstrate the superior performance of the proposed nested two-stage MAB strategy.
• The nested two-stage TS approach achieves performance close to optimal.
• Codebook-based phase shifters effectively overcome CSI estimation difficulties.

Conclusions:

• The proposed nested two-stage MAB strategy offers an effective solution for optimizing RIS and mmWave TX phase shifts.
• This approach significantly enhances mmWave communication coverage.
• The use of MAB algorithms, particularly TS, provides a robust and near-optimal performance in dynamic communication environments.