VolGen: Volumetric Latent Diffusion Models for 3D Object Generation.

Area of Science:

• Computer Vision
• 3D Deep Learning
• Generative Models

Background:

• 2D latent diffusion models, like Stable Diffusion, excel at image generation.
• Generating high-fidelity 3D objects from limited input remains a challenge.
• Existing 3D generation methods often rely on complex representations.

Purpose of the Study:

• To extend 2D latent diffusion models to the 3D domain for object generation.
• To develop a computationally efficient method for 3D mesh generation.
• To demonstrate the effectiveness of volume-based latent diffusion for 3D tasks.

Main Methods:

• Training a Volumetric Variational Auto-Encoder (VVAE) to compress 3D occupancy grids into a compact latent space.
• Developing a diffusion model that operates on this latent space using 3D convolutions and cross-attention.
• Implementing a Volumetric Latent Diffusion Model (VLDM) for single-view 3D object reconstruction.

Main Results:

• The VLDM successfully generates accurate and smooth 3D mesh surfaces from single-view images.
• The model achieves efficient inference, completing generation in approximately 10 seconds.
• The approach demonstrates strong generalization capabilities to unseen domains.

Conclusions:

• Volume-based latent diffusion models are effective for 3D generation tasks.
• This method offers a promising alternative to sparse representations and specialized 3D techniques.
• The VLDM represents a significant advancement in efficient and accurate 3D object synthesis.