Structure-From-Motion in Spherical Video Using the von Mises-Fisher Distribution

Area of Science:

• Computer Vision
• Robotics
• 3D Reconstruction

Background:

• Structure-from-motion (SfM) is crucial for 3D scene reconstruction from images.
• Existing SfM methods are primarily designed for perspective cameras, limiting their application to spherical imagery.
• Spherical images capture a full 360-degree field of view, presenting unique geometric challenges.

Purpose of the Study:

• To develop a comprehensive pipeline for structure-from-motion (SfM) specifically tailored for spherical image sequences.
• To adapt and propose novel solutions for multiview geometry problems within the spherical camera domain.
• To improve the accuracy and robustness of 3D reconstruction from omnidirectional visual data.

Main Methods:

• Developed specialized methods for the spherical n-point problem (camera pose estimation) and calibrated spherical reconstruction (3D point localization).
• Introduced a probabilistic model for spherical feature point noise using the von Mises-Fisher distribution.
• Integrated this probabilistic model into bundle adjustment for enhanced optimization.

Main Results:

• Quantitative and qualitative evaluations on synthetic and real-world data demonstrate superior performance of spherical SfM methods over perspective adaptations.
• The proposed probabilistic model and bundle adjustment improved SfM accuracy for spherical images.
• Successful application in stabilizing viewing direction for fully spherical video.

Conclusions:

• The developed pipeline and methods provide an effective solution for structure-from-motion with spherical images.
• Specialized spherical geometry approaches are essential for accurate 3D reconstruction from omnidirectional data.
• This work advances SfM capabilities for applications involving 360-degree imaging and immersive media.