DeepSFM: Robust Deep Iterative Refinement for Structure From Motion.

Area of Science:

• Computer Vision
• Deep Learning
• Robotics

Background:

• Structure from Motion (SfM) is crucial for 3D reconstruction but challenging for deep learning.
• Accurate camera pose estimation from images alone is difficult due to environmental factors.
• Existing methods often rely on unrealistic assumptions of perfect camera poses.

Purpose of the Study:

• To develop a novel deep learning architecture for Structure from Motion (SfM).
• To address the limitations of current SfM methods by incorporating explicit structural constraints.
• To improve the accuracy and robustness of camera pose and depth estimation.

Main Methods:

• A physically driven architecture, DeepSFM, inspired by Bundle Adjustment (BA).
• Utilizes two cost volume-based networks for iterative depth and pose refinement.
• Incorporates Gated Recurrent Units (GRUs) for efficient iterative updates and residual depth prediction for dynamic scenes.

Main Results:

• Achieves state-of-the-art performance on various datasets.
• Demonstrates superior robustness against challenging environmental factors and inputs.
• Effectively adapts to dynamic scenes through residual depth prediction.

Conclusions:

• DeepSFM successfully combines traditional Bundle Adjustment principles with deep learning.
• The explicit depth and pose constraints enhance SfM accuracy and robustness.
• The proposed model offers an efficient and adaptable solution for Structure from Motion problems.