DP-SfM: Dual-Pixel Structure-From-Motion Without Scale Ambiguity

Area of Science:

• Computer Vision
• 3D Reconstruction
• Computational Photography

Background:

• Multi-view 3D reconstruction (structure-from-motion and multi-view stereo) is crucial for 3D computer vision.
• Scale ambiguity is a common challenge in multi-view 3D reconstruction, often requiring known reference objects or prior calibration.
• Dual-pixel (DP) sensors offer unique imaging properties due to their on-sensor phase-detection autofocus pixels.

Purpose of the Study:

• To demonstrate that multi-view images from dual-pixel sensors can resolve scale ambiguity in 3D reconstruction.
• To develop a method for absolute scale estimation using DP image defocus blur without external references.
• To improve the accuracy and applicability of multi-view 3D reconstruction techniques.

Main Methods:

• Utilizing defocus blur information from dual-pixel images to determine absolute scale.
• Integrating scale estimation with existing depth maps (up to scale) from multi-view 3D reconstruction.
• Developing a linear method for absolute scale estimation followed by cross-view blur kernel optimization for image alignment.

Main Results:

• Successfully resolved scale ambiguity in multi-view 3D reconstruction using only dual-pixel images.
• Achieved accurate absolute scale estimation without needing reference objects or camera calibration.
• Demonstrated the method's effectiveness across various scenes and camera/lens configurations.

Conclusions:

• Dual-pixel sensor technology inherently provides the necessary information to overcome scale ambiguity in 3D reconstruction.
• The proposed method offers a simple, effective, and automatic solution for absolute scale determination in structure-from-motion and multi-view stereo.
• This work significantly advances the practical application of multi-view 3D reconstruction in computer vision.