Hear What You See: Depth-Aware Spatial Audio Rendering for Immersive Panoramic Video

Panoramic video has emerged as a core medium for virtual reality (VR) and immersive media, yet its audio typically remains stereo, lacking spatially coherent soundfield reproduction consistent with the visual content and severely compromising immersion. Although multi-loudspeaker spatial audio systems can reconstruct realistic soundfields in physical spaces, existing solutions remain heavily reliant on manual production, and mainstream algorithms such as Vector Base Amplitude Panning (VBAP) handle only directional information while inadequately modeling distance perception. This paper proposes the Vision-Acoustic Coupled Spatialization (VACS) framework, a calibration-assisted, semi-automatic pipeline that re-spatializes prepared audio from panoramic video: given user-designated sounding targets, prepared or pre-separated audio streams, and near-far depth references, it extracts target motion trajectories and calibrated relative-depth controls and generates spatially consistent multichannel audio. The core innovation is the Depth-Aware Spatial Audio Rendering (DASAR) method, which leverages monocular depth estimation to integrate sound pressure level (SPL) attenuation, spectral adjustment, and reverberation control within a unified framework, overcoming the angle-only limitation of conventional VBAP. Experiments show 62.6% localization accuracy within 10° tolerance, with Pearson correlations of 0.877 and 0.843 ($p$ < 0.001) for azimuth and elevation. Subjectively, compared with VBAP, spatial depth and immersion improve by 13.2% ($p$ < 0.05) and 19.5% ($p$ < 0.01), respectively. Under the tested protocol, no statistically significant difference from manual production was observed on the three measured dimensions of localization accuracy, spatial depth, and immersion ($p$ > 0.05), while reducing manual trajectory and distance-parameter editing.