FoVolNet: Fast Volume Rendering using Foveated Deep Neural Networks

Area of Science:

• Computer Graphics
• Scientific Visualization
• Artificial Intelligence

Background:

• Volume data visualization is crucial for scientific and engineering applications.
• High-quality, interactive rendering for demanding applications like virtual reality remains a challenge.
• Existing methods struggle to balance performance and visual fidelity.

Purpose of the Study:

• To introduce FoVolNet, a novel method for significantly improving volume data visualization performance.
• To develop a cost-effective foveated rendering pipeline combined with deep neural network reconstruction.
• To achieve fast, stable, and perceptually convincing volume rendering for interactive applications.

Main Methods:

• Implemented a foveated rendering pipeline that sparsely samples volume data around a focal point.
• Developed a deep neural network for reconstructing the full frame from sparsely sampled data.
• Combined direct and kernel prediction methods within the reconstruction network, incorporating quantization for efficiency.

Main Results:

• FoVolNet achieves significant time savings over conventional rendering methods.
• The method preserves perceptual quality, delivering convincing visual output.
• Outperforms state-of-the-art neural reconstruction techniques in end-to-end frame times and visual quality.

Conclusions:

• FoVolNet offers a substantial performance increase for volume data visualization.
• The approach effectively leverages human visual system properties for computational savings.
• Presents a viable solution for high-quality, interactive volume rendering in demanding applications.