Templateless Non-Rigid Reconstruction and Motion Tracking With a Single RGB-D Camera

Area of Science:

• Computer Vision
• 3D Reconstruction
• Robotics

Background:

• Accurate 3D reconstruction and motion tracking of deformable objects are challenging.
• Existing methods often rely on template priors, limiting their applicability.
• Dynamic scenes with significant deformations require robust registration techniques.

Purpose of the Study:

• To develop a novel templateless approach for nonrigid reconstruction and motion tracking.
• To achieve accurate 3D reconstruction and tracking of considerably deformable objects without template priors.
• To enable robust registration of dynamic motion sequences from partial depth scans.

Main Methods:

• A local-to-global hierarchical optimization framework inspired by structure-from-motion.
• Local nonrigid bundle adjustment for segment alignment to prevent error accumulation.
• Global optimization with loop-closure constraints to address drift and combine segments.

Main Results:

• The proposed framework successfully reconstructs and tracks deforming 3D object sequences.
• Demonstrated robustness and efficiency in handling considerable motions on synthetic and real data.
• Achieved high-quality, drift-free 3D model sequences without relying on template priors.

Conclusions:

• The novel templateless approach offers accurate and robust nonrigid reconstruction and motion tracking.
• The local-to-global optimization framework effectively handles dynamic deformations and loop sequences.
• This method advances 3D reconstruction capabilities for deformable objects in computer vision and robotics.