Isometric Non-Rigid Shape-from-Motion with Riemannian Geometry Solved in Linear Time

Area of Science:

• Computer Vision
• 3D Reconstruction
• Differential Geometry

Background:

• Non-rigid shape-from-motion (NRSfM) aims to reconstruct 3D shapes from 2D images.
• Isometric deformations preserve distances and angles, posing unique challenges for reconstruction.
• Existing methods struggle with simultaneously estimating first and second-order derivatives of inverse-depth for Iso-NRSfM.

Purpose of the Study:

• To develop a novel theoretical framework for Isometric Non-Rigid Shape-from-Motion (Iso-NRSfM).
• To propose an accurate and computationally efficient iterative solver for Iso-NRSfM.
• To reconstruct time-varying 3D shapes of thin-shell objects undergoing isometric deformations.

Main Methods:

• Utilized a Riemannian manifold framework to represent 3D surfaces as embeddings of the camera's retinal plane.
• Employed metric tensor and Christoffel Symbol (CS) fields derived from inverse-depth derivatives.
• Developed a two-step iterative solver: first estimating first-order derivatives, then second-order derivatives, iterating until convergence.

Main Results:

• Established theoretical results relating metric tensor and CS across images based on local warps.
• Formulated a globally solvable system of cubics for first-order derivatives (N>=3 images).
• Achieved superior accuracy and reduced computation cost compared to existing methods on synthetic and real datasets.

Conclusions:

• The proposed Riemannian manifold approach effectively addresses Iso-NRSfM.
• The iterative two-step solver provides a robust and efficient solution for 3D shape reconstruction.
• The method demonstrates significant improvements in accuracy and efficiency for dynamic shape recovery.