Articulated Motion-Aware NeRF for 3D Dynamic Appearance and Geometry Reconstruction by Implicit Motion States

Area of Science:

• Computer Vision
• 3D Reconstruction
• Machine Learning

Background:

• Reconstructing dynamic 3D scenes with articulated objects from images is challenging.
• Existing methods often struggle with discontinuous or unordered image sets and capturing continuous motion attributes.

Purpose of the Study:

• To develop a self-supervised approach for 3D dynamic reconstruction of articulated motions.
• To recover continuous motions and attributes from unordered, discontinuous image data.
• To achieve high-fidelity reconstruction of appearance, geometry, and motion.

Main Methods:

• Utilized Generative Adversarial Networks (GANs) to create a continuous implicit motion state space.
• Employed a motion network to extract discrete motion states as anchors, expanded via GANs.
• Integrated motion-aware Neural Radiance Fields for dynamic appearance and geometry reconstruction.
• Applied a cluster-based strategy to deduce motion attributes from continuous motions.

Main Results:

• Successfully reconstructed articulated objects and their continuous motions from image sets.
• Demonstrated superior fidelity in appearance, geometry, and motion attributes compared to state-of-the-art methods.
• Validated the approach on both synthesized and real-world data.

Conclusions:

• The proposed self-supervised method effectively reconstructs dynamic articulated motions.
• Treating motion states as time-independent enables robust reconstruction from discontinuous data.
• The approach offers a significant advancement in 3D dynamic reconstruction for articulated objects.