HandNeRF++: Modeling Animatable Interacting Hands With Neural Radiance Fields.

Area of Science:

• Computer Vision
• Computer Graphics
• Machine Learning

Background:

• Reconstructing accurate 3D models of interacting hands from images is challenging due to complex occlusions and self-occlusions.
• Existing methods struggle with texture contamination and shape artifacts in intricate hand interactions.

Purpose of the Study:

• To develop a robust framework for photo-realistic free-viewpoint animation of interacting hands.
• To address limitations in existing Neural Radiance Fields (NeRF)-based approaches for hand reconstruction.

Main Methods:

• Introduced HandNeRF++, an advanced NeRF-based framework utilizing a pose-driven deformation field for canonical space mapping.
• Implemented pose- and shape-disentangled NeRFs, adaptive hand-sharing for cross-hand augmentation, and pose-guided density priors for occlusion-aware learning.
• Employed neural feature distillation and a neural refiner for color optimization and rendering enhancement.

Main Results:

• HandNeRF++ significantly improves the accuracy of appearance and geometry reconstruction for interacting hands.
• The framework effectively overcomes texture contamination and shape artifacts in complex hand scenarios.
• Achieved state-of-the-art qualitative and quantitative results in free-viewpoint hand pose animation.

Conclusions:

• HandNeRF++ represents a significant advancement in NeRF-based 3D reconstruction for interacting hands.
• The proposed methods enhance detail, handle occlusions better, and produce more realistic animations.
• Validated through extensive experiments, demonstrating superior performance in hand animation rendering.