Neural Vector Fields: Generalizing Distance Vector Fields by Codebooks and Zero-Curl Regularization.

Area of Science:

• Computer Vision
• Computer Graphics
• Machine Learning

Background:

• Neural network-based 3D surface reconstruction methods typically fall into explicit template warping or implicit surface representation categories.
• Existing methods often face limitations in resolution, topology, and require complex surface extraction processes.

Purpose of the Study:

• To propose a novel 3D representation, Neural Vector Fields (NVF), that integrates the strengths of explicit and implicit surface reconstruction techniques.
• To develop a method that overcomes resolution and topological barriers in 3D surface reconstruction.
• To introduce a differentiation-free approach for calculating direction fields, simplifying surface extraction.

Main Methods:

• Introduced Neural Vector Fields (NVF) by directly predicting displacements from surface queries and modeling shapes as Vector Fields.
• Developed NVFs (Lite and Ultra) incorporating shape codebooks for cross-category reconstruction by encoding cross-object priors.
• Proposed a novel regularization based on the zero-curl property of NVFs, implemented in a fully differentiable framework for NVF (Ultra).

Main Results:

• NVFs enable direct prediction of displacements and modeling shapes as Vector Fields, bypassing the need for network differentiation to obtain direction fields.
• The proposed approach effectively encodes both distance and direction fields, circumventing complex surface extraction steps.
• Evaluated NVFs on diverse scenarios including watertight/non-watertight shapes, category-agnostic/unseen reconstruction, category-specific, and cross-domain reconstruction.

Conclusions:

• Neural Vector Fields offer a robust and versatile 3D representation for surface reconstruction, combining explicit and implicit learning advantages.
• The differentiation-free direction field calculation and shape codebook integration enhance reconstruction capabilities across various scenarios.
• The zero-curl regularization further refines NVF performance, demonstrating significant potential for advanced 3D shape modeling and reconstruction.