DeSC: Learning Deep Semantic Descriptor for NeRF Registration

Area of Science:

• Computer Vision
• 3D Reconstruction
• Machine Learning

Background:

• Neural Radiance Field (NeRF) registration is a growing research area.
• Existing methods often focus on geometric or photometric information, neglecting cross-modal features within NeRF embeddings.
• This limitation hinders robust feature learning for accurate scene alignment.

Purpose of the Study:

• To propose DeSC, a novel approach for NeRF registration.
• To leverage rich cross-modal features from NeRF embeddings for robust semantic descriptor learning.
• To improve alignment accuracy and robustness in NeRF registration.

Main Methods:

• Introduced a Deep Semantic Aggregation module utilizing a weighted graph convolution network.
• Captured high-frequency texture details within NeRF patches to reveal shared semantics across different NeRFs.
• Incorporated a density-aware photometric consistency loss to enhance feature learning.

Main Results:

• DeSC effectively learns robust global feature descriptors by exploiting cross-modal information.
• The approach demonstrated superior registration performance compared to state-of-the-art techniques on Objaverse datasets.
• Experimental results validated improved alignment accuracy and robustness.

Conclusions:

• DeSC offers a novel and effective method for NeRF registration by utilizing cross-modal features.
• The proposed Deep Semantic Aggregation module and density-aware loss contribute to robust semantic descriptor learning.
• This work advances the field of NeRF registration, providing a more accurate and robust alignment solution.