Generating Audiovisual Synergy Fluid Animation for Highly Immersive VR Experience

Area of Science:

• Computer Graphics
• Virtual Reality
• Fluid Dynamics Simulation

Background:

• Generative content in VR aims for immersive experiences, but audiovisual effects, especially fluid phenomena, face quality challenges.
• Realism and presence in dynamic fluid animations require adherence to physical laws for effective immersion.

Purpose of the Study:

• To propose an audiovisual synergy fluid animation generation framework to enhance VR immersion.
• To improve motion texture fidelity and audiovisual consistency in generated fluid animations.

Main Methods:

• Developed a Detail-Enhanced Texture generator (DET) using Global-Local Physics guidance (GLP) and Temporal Texture Modeling (TTM).
• Created a Physics-Guided Audio generator (PGA) with Visual Semantic Augmenter (VSA) and Rhythm Semantic Adapter (RSA) for audio-visual synchronization.
• Integrated DET and PGA to optimize dynamic details and temporal coherence in fluid animations.

Main Results:

• The framework significantly enhances audiovisual immersion in VR natural dynamic scenes.
• Quantitative and qualitative evaluations show superior performance over existing methods in texture realism and audiovisual synchronization.
• The approach offers new insights for advancing immersive experiences in dynamic VR phenomena.

Conclusions:

• The proposed framework effectively generates high-quality, physically plausible fluid animations for VR.
• Improved texture fidelity and audiovisual synchronization lead to greater realism and presence in virtual environments.
• This work advances the state-of-the-art in immersive dynamic VR content generation.