LES-derived turbulence tank optical propagation with numerical adaptive-optics modeling: quantifying OAM modal crosstalk and mitigation

Orbital angular momentum (OAM) modes can increase the capacity of free-space optical communication, but turbulence-induced modal crosstalk limits their practical use. Atmospheric turbulence simulators offer controlled test environments; however, their short propagation distances make it unclear how convection-driven spatial heterogeneity and weak scintillation affect OAM spectra and adaptive-optics (AO) correction. Here, we investigate the propagation of vortex beams through Rayleigh-Benard (RB) turbulence and evaluate AO compensation under physically controlled conditions. The RB turbulence produces strong phase distortions with weak scintillation, leading to progressive OAM modal degradation without pronounced bimodal features in the power spectrum. As the vertical temperature difference increases, modal power is redistributed into an hourglass-shaped pattern associated with the large-scale circulation of RB convection, with stronger perturbations near the boundaries and weaker perturbations in the central region. We further implement a hardware-parameterized AO framework that incorporates wavefront sensing, phase reconstruction, and deformable-mirror correction. The results show that AO effectively suppresses modal crosstalk and improves recovery of the OAM power spectrum under boundary-driven turbulence. These findings clarify how convective flow structures perturb OAM modes and provide quantitative guidance for assessing turbulence-compensation strategies in OAM-based free-space optical communication.