Saliency-Guided Foveated Video Encoding for Low-Latency and Immersive Cloud VR

In cloud virtual reality (VR), delivering high perceptual visual quality under constrained wireless bandwidth remains a pivotal challenge. Foveated video encoding (FVE) copes with this by human vision-driven quality allocation, reducing bandwidth demand while preserving high visual fidelity for immersive VR streaming. However, existing FVE methods are content-independent, relying exclusively on gaze position and predefined quality degradation profiles. Thus, they fall short in modeling the intricate mechanisms of human attention. This leads to suboptimal quality distribution for visual saliency stimuli, causing perceptible artifacts and degraded perceptual quality. In this paper, we introduce a novel AI-driven streaming framework that incorporates visual saliency cues, defined as the innate capacity of scene elements to attract attention, into the video encoding pipeline. To meet the stringent low-latency demands of immersive VR, we propose a lightweight deep neural network for saliency inference, reducing computational complexity (FLOPs) by 48× compared to prior models while maintaining comparable accuracy. We integrate our pipeline into an open-source cloud VR gaming platform and conduct comprehensive experiments. Evaluation results demonstrate that our approach enhances perceptual visual quality by 22.98% compared to SoTA systems. Our IRB-approved user study shows that the saliency-guided FVE pipeline achieves superior visual quality and spatial smoothness while significantly reducing noticeable artifacts introduced by gaze-exclusive FVE. The project source code is available at https://github.com/WuZemyp/SaliencyFov.