NeuMesh++: Toward Versatile and Efficient Volumetric Editing With Disentangled Neural Mesh-Based Implicit Field.

Area of Science:

• Computer Vision
• Computer Graphics
• Artificial Intelligence

Background:

• Neural implicit rendering excels at novel view synthesis and 3D reconstruction.
• Current neural rendering methods have limited editing capabilities, restricting users to rigid transformations or category-specific edits.

Purpose of the Study:

• To develop a novel mesh-based neural rendering representation for comprehensive 3D scene editing.
• To enable efficient and versatile editing functionalities beyond existing methods.

Main Methods:

• Encoding neural radiance fields with disentangled geometry, texture, and semantic codes on mesh vertices.
• Developing techniques including local space parameterization, learnable vertex color modification, spatial-aware optimization, and semantic-aided region selection.

Main Results:

• Achieved efficient and comprehensive editing, including mesh-guided geometry editing, texture swapping, filling, painting, and semantic-guided editing.
• Demonstrated superior representation quality and editing ability on real and synthetic datasets.

Conclusions:

• The proposed mesh-based neural rendering method offers powerful and flexible editing capabilities for 3D scenes.
• The approach significantly advances the state-of-the-art in neural rendering for interactive scene manipulation.