HomuGAN: A 3D-aware GAN with the Method of Cylindrical Spatial-Constrained Sampling

Area of Science:

• Computer Vision
• Artificial Intelligence
• 3D Graphics

Background:

• Controllable 3D-aware scene synthesis aims for disentangled latent codes in implicit spaces for realistic, 3D-consistent image generation.
• Current methods often combine Neural Radiance Fields with StyleGAN2's upsampling, using convolutions for spatial-to-frequency transformation.
• This integration can lead to a 'bubble phenomenon,' degrading 3D implicit modeling and image quality due to extraneous information from style modulation.

Purpose of the Study:

• To address the limitations of existing 3D-aware scene synthesis methods, specifically the 'bubble phenomenon' and training instability.
• To introduce a novel generative model, HomuGAN, that enhances disentanglement and 3D consistency.
• To improve the quality and stability of generating realistic 3D scenes.

Main Methods:

• HomuGAN disentangles style modulation for implicit modeling from super-resolution.
• Introduces Cylindrical Spatial-Constrained Sampling and Parabolic Sampling methods.
• Parabolic Sampling is optimized for foreground vehicle modeling.

Main Results:

• HomuGAN effectively alleviates the 'bubble phenomenon' by separating style modulation applications.
• Achieves state-of-the-art performance on public datasets with superior disentanglement capabilities.
• Demonstrates improved training stability compared to StyleNeRF.

Conclusions:

• HomuGAN offers a significant advancement in controllable 3D-aware scene synthesis.
• The proposed disentanglement strategy and novel sampling methods enhance realism and consistency.
• HomuGAN provides a more stable and effective solution for generating high-quality 3D scenes.