GeoStyler: A Generalizable Geometry-Aware Diffusion-Based Approach for Direct 3D Gaussian Style Transfer.

Area of Science:

• Computer Vision
• Computer Graphics
• Artificial Intelligence

Background:

• Direct 3D scene stylization from sparse views is challenging due to slow optimization methods and geometric corruption.
• Existing direct methods often cause visual artifacts by rigidly decoupling geometry and appearance.

Purpose of the Study:

• To introduce GeoStyler, a direct framework for high-fidelity, multi-view consistent stylized 3D scenes.
• To accelerate the 3D scene stylization process while maintaining geometric integrity.

Main Methods:

• Leveraging a diffusion model with a novel hybrid query formulation in self-attention to generate geometrically consistent stylized 2D images.
• Embedding cross-view geometric information into queries and style information into keys/values for 3D consistency and structure preservation.
• Utilizing a geometrically-aware latent initialization and a decoupled reconstruction network for 3D Gaussian representation.

Main Results:

• GeoStyler generates stylized 3D scenes in seconds with high fidelity and multi-view consistency.
• Achieved state-of-the-art performance on large-scale benchmarks like RealEstate10K and ACID.
• Demonstrated significant speedup compared to previous methods in stylization quality.

Conclusions:

• GeoStyler effectively addresses limitations of previous 3D scene stylization techniques.
• The framework successfully balances stylization quality with geometric structural integrity.
• Offers a fast and robust solution for direct 3D scene stylization.