Editing 3D Scenes via Text Prompts Without Retraining.

Area of Science:

• Computer Vision
• 3D Graphics
• Artificial Intelligence

Background:

• Diffusion models have advanced 2D image synthesis and editing.
• Extending these to 3D scene editing faces challenges in scene representation and multi-view consistency.
• Existing methods often require scene-specific modules and retraining for each editing task.

Purpose of the Study:

• To propose a versatile text-driven 3D scene editing method (DN2N).
• To enable direct editing of 3D scenes without model retraining.
• To address multi-view consistency issues in text-driven 3D scene editing.

Main Methods:

• Utilizes off-the-shelf 2D text-based editing models on multi-view images of a 3D scene.
• Applies content filtering to maintain 3D consistency.
• Develops a versatile Neural Radiance Fields (NeRF) model structure with novel cross-view regularization terms to mitigate perturbations.

Main Results:

• Demonstrates multiple editing types via text prompts, including appearance changes, weather transitions, object replacement, and style transfer.
• Achieves versatile editing capabilities without scene- or edit-type-specific model customization or retraining.
• Shows comparable editing time to existing 3D Gaussian Splatting (3DGS)-based methods, improving practical applicability.

Conclusions:

• DN2N offers a versatile and efficient solution for text-driven 3D scene editing.
• The method overcomes limitations of existing approaches by eliminating the need for retraining and specialized modules.
• DN2N enhances the practical value of 3D scene editing through its broad applicability and comparable performance.