AgonicDreamer: Enhancing Multi-View Consistency in Text-to-3D Generation via Rectified Score Distillation

Area of Science:

• Computer Vision
• 3D Graphics
• Artificial Intelligence

Background:

• Score Distillation Sampling (SDS) methods leverage pre-trained text-to-image diffusion models for text-to-3D generation.
• Existing SDS approaches struggle with view-inconsistency, commonly known as the Janus problem, due to view-agnostic score estimation.

Purpose of the Study:

• To introduce a novel framework, AgonicDreamer, that generates multi-view consistent and photorealistic 3D content.
• To address the limitations of view-agnostic score estimation in current text-to-3D generation techniques.

Main Methods:

• Propose Rectified Score Distillation (RSD) by formulating a view-conditioned reverse ordinary differential equation (ODE) based on camera poses.
• Rectify standard, view-irrelevant scores to approximate gradients along the proposed view-conditioned ODE.
• Integrate rectified scores into a comprehensive framework for optimizing 3D representations.

Main Results:

• AgonicDreamer successfully generates high-fidelity 3D assets with improved multi-view consistency.
• The proposed Rectified Score Distillation effectively mitigates the Janus problem observed in previous methods.
• Extensive experiments validate the framework's capability in producing photorealistic 3D content with fine-grained details.

Conclusions:

• Rectified Score Distillation offers a robust solution for achieving view-consistent text-to-3D generation.
• AgonicDreamer represents a significant advancement in generating high-quality, consistent 3D content from textual descriptions.
• The method provides a promising direction for future research in generative 3D modeling.