Accurate 3D Reconstruction from Small Motion Clip for Rolling Shutter Cameras

Area of Science:

• Computer Vision
• 3D Reconstruction
• Computational Imaging

Background:

• Structure from motion (SfM) is crucial for 3D computer vision and depth estimation.
• Limitations in SfM include depth uncertainty caused by narrow baselines and rolling shutter effects inherent in consumer cameras.
• Existing methods struggle with accurate dense 3D reconstruction from small motion sequences with these artifacts.

Purpose of the Study:

• To develop a dense 3D reconstruction method robust to small motion, narrow baselines, and rolling shutter artifacts.
• To introduce novel techniques for compensating rolling shutter effects in 3D reconstruction.
• To enable user-friendly 3D data capture using readily available hand-held cameras.

Main Methods:

• A novel small motion bundle adjustment is proposed to compensate for rolling shutter distortions.
• A fine-scale dense 3D reconstruction pipeline is developed, modeling rolling shutter effects.
• The method utilizes sparse 3D points and camera trajectory from narrow-baseline images, propagating depth hypotheses using geometry guidance.

Main Results:

• The proposed framework achieves accurate dense 3D reconstruction results.
• The method effectively models and compensates for rolling shutter artifacts.
• Qualitative and quantitative evaluations demonstrate superior depth map generation compared to state-of-the-art techniques.

Conclusions:

• The developed method provides accurate and dense 3D reconstructions from small motion sequences captured with hand-held cameras.
• The novel bundle adjustment and reconstruction pipeline effectively address limitations of narrow baselines and rolling shutter effects.
• This approach offers a significant advancement for 3D computer vision applications requiring user-friendly input and high-quality depth estimation.