Deep Non-Rigid Structure From Motion With Missing Data.

Area of Science:

• Computer Vision
• Machine Learning
• 3D Reconstruction

Background:

• Non-rigid structure from motion (NRSfM) algorithms face limitations in handling the number of images and shape variability.
• Existing methods struggle with the trade-off between system conditions and modeling degrees of freedom, limiting practical applications.

Purpose of the Study:

• To develop a novel hierarchical sparse coding model for NRSfM that overcomes current limitations.
• To enable NRSfM applications previously considered too ill-posed due to complexity.

Main Methods:

• Proposed a hierarchical sparse coding model implemented as an unsupervised deep neural network (DNN) auto-encoder.
• The DNN architecture disentangles pose from 3D structure without 3D supervision, using only 2D point correspondences.
• The approach handles missing or occluded 2D points without matrix completion.

Main Results:

• Achieved NRSfM at unprecedented scale and shape complexity using deep learning platforms.
• Demonstrated superior precision and robustness compared to state-of-the-art methods, in some cases by an order of magnitude.
• Introduced a new quality measure based on network weights to assess reconstructability without 3D ground-truth.

Conclusions:

• The proposed DNN-based NRSfM approach represents a significant advancement over existing state-of-the-art methods.
• The model's ability to handle complex shape variations and missing data broadens the applicability of NRSfM.
• The novel quality measure offers a reliable way to evaluate reconstruction confidence.