StyleRF-VolVis: Style Transfer of Neural Radiance Fields for Expressive Volume Visualization

Area of Science:

• Computer Graphics
• Scientific Visualization
• Artificial Intelligence

Background:

• Volume visualization (VolVis) synthesis generates novel visualizations beyond traditional rendering.
• Existing generative adversarial network (GAN) methods suffer from long training times, low quality, and inconsistency.
• Neural radiance fields (NeRF) offer a promising avenue for advanced visualization techniques.

Purpose of the Study:

• Introduce StyleRF-VolVis, a novel style transfer framework for expressive volume visualization.
• Enable separation of scene geometry (content) and color appearance (style) in 3D scenes.
• Provide flexible control over color, opacity, and lighting while maintaining visual consistency.

Main Methods:

• Developed a base NeRF model for scene geometry extraction.
• Designed a palette color network for photorealistic style editing.
• Implemented an unrestricted color network with knowledge distillation for non-photorealistic editing.
• Utilized neural radiance fields (NeRF) for expressive volume visualization.

Main Results:

• StyleRF-VolVis accurately separates content and style in volume data.
• The framework allows for convenient modification of visual attributes like color and opacity.
• Demonstrated superior quality, consistency, and flexibility compared to existing methods (AdaIN, ReReVST, ARF, SNeRF).
• Successfully transferred arbitrary styles to reconstructed 3D scenes.

Conclusions:

• StyleRF-VolVis provides an effective and flexible approach to expressive volume visualization.
• The proposed method overcomes limitations of previous GAN-based and NeRF-based style transfer techniques.
• StyleRF-VolVis advances the field of neural rendering for scientific visualization.