MicroDreamer: Efficient 3D Generation in $\sim$20 Seconds by Score-based Iterative Reconstruction.

Area of Science:

• Computer Vision
• 3D Computer Graphics
• Machine Learning

Background:

• Optimization-based methods like score distillation sampling (SDS) are promising for zero-shot 3D generation but are inefficient due to high function evaluations and latent space limitations.
• Existing methods struggle with balancing generation speed and quality in 3D reconstruction tasks.

Purpose of the Study:

• To introduce an efficient and general algorithm, score-based iterative reconstruction (SIR), for 3D generation.
• To enable optimization in pixel space and reduce the number of function evaluations (NFEs) compared to traditional SDS methods.
• To present MicroDreamer, an efficient approach for various 3D representations and generation tasks.

Main Methods:

• Developed score-based iterative reconstruction (SIR), an algorithm that mimics differentiable 3D reconstruction.
• Implemented iterative optimization in pixel space, contrasting with SDS's single-step latent space optimization.
• Integrated SIR with multi-view score-based diffusion models and optimized training for enhanced performance.

Main Results:

• MicroDreamer demonstrates 5-20 times speed improvement over SDS for neural radiance field generation with comparable quality.
• Achieved mesh generation from 3D Gaussian splatting in approximately 20 seconds on a single A100 GPU.
• Outperformed the baseline DreamGaussian by a significant margin while halving generation time.

Conclusions:

• MicroDreamer offers a substantial advancement in efficient and high-quality zero-shot 3D generation.
• The SIR algorithm provides a general and effective framework applicable to diverse 3D representations and tasks.
• This work significantly reduces computational costs and time for 3D content creation.