Scene-aware Foveated Rendering.

Area of Science:

• Computer Graphics
• Human-Computer Interaction
• Image Processing

Background:

• Foveated rendering optimizes rendering by allocating more detail to the human visual system's high-acuity foveal region.
• Existing mapping-based foveated rendering methods can be improved by incorporating scene awareness and human visual characteristics.

Purpose of the Study:

• To develop a scene-aware foveated rendering method that improves perceptual quality and visual fidelity.
• To integrate scene features, visual acuity, and gaze motion into the rendering process.

Main Methods:

• Generating a conservative visual importance map encoding scene features, visual acuity, and gaze motion.
• Constructing a pixel size control map using a convolution kernel method.
• Guiding foveated rendering with the pixel size control map and applying temporal coherent refinement.

Main Results:

• The proposed method outperforms state-of-the-art mapping-based foveated rendering in terms of Mean Squared Error (MSE), Peak Signal-to-Noise Ratio (PSNR), and Structural Similarity Index Measure (SSIM).
• Improved metrics were observed in the fovea, periphery, salient regions, and overall image.
• User studies confirmed high visual similarity between the rendered output and ground truth at full resolution.

Conclusions:

• The scene-aware foveated rendering method effectively enhances visual quality and perceptual fidelity.
• Integrating scene awareness and human visual system characteristics leads to superior rendering performance compared to existing techniques.