ConsDreamer: Advancing Multi-View Consistency for Zero-Shot Text-to-3D Generation

Area of Science:

• Computer Vision
• Computer Graphics
• Artificial Intelligence

Background:

• Zero-shot text-to-3D generation enables direct 3D content creation from text.
• Current methods use 3D Gaussian Splatting and score distillation with text-to-image (T2I) models.
• Inherent view biases in T2I models cause inconsistent 3D generation, notably the multi-face Janus problem.

Purpose of the Study:

• To propose a novel method, ConsDreamer, to mitigate view bias in text-to-3D generation.
• To address the multi-face Janus problem by refining score distillation terms.
• To enhance geometric consistency and accuracy in generated 3D models.

Main Methods:

• ConsDreamer refines both conditional and unconditional terms in score distillation.
• A View Disentanglement Module (VDM) removes viewpoint biases from conditional prompts.
• A similarity-based partial order loss enforces geometric consistency in the unconditional term.

Main Results:

• ConsDreamer effectively mitigates the multi-face Janus problem.
• The method demonstrates seamless integration with various 3D representations.
• Experiments confirm improved geometric consistency in generated 3D objects.

Conclusions:

• ConsDreamer offers a robust solution to view bias in text-to-3D generation.
• The proposed approach enhances the reliability and consistency of 3D content creation.
• This work advances the field of generative 3D modeling by addressing a fundamental limitation.