Capturing the Geometry of Object Categories from Video Supervision

Area of Science:

• Computer Vision
• Machine Learning
• 3D Geometry

Background:

• Traditional 3D reconstruction often relies on precise CAD models or extensive manual annotation.
• Learning object geometry from uncalibrated video data presents significant challenges due to viewpoint variations and occlusions.

Purpose of the Study:

• To develop an unsupervised deep learning framework for inferring 3D object geometry from uncalibrated video sequences.
• To enable prediction of object viewpoint, depth, and complete 3D shape from single images without requiring manual supervision or CAD models.

Main Methods:

• A novel deep neural network architecture comprising three components: Siamese viewpoint factorization, monocular depth estimation, and shape completion.
• Leveraging probabilistic predictions across all network modules to enhance robustness and enable self-assessment.
• Training on video sequences of object instances from moving viewpoints.

Main Results:

• Achieved state-of-the-art performance on public benchmarks for viewpoint prediction, depth estimation, and 3D point cloud generation.
• Demonstrated the effectiveness of probabilistic predictions for improving the quality of 3D geometry inference.
• Successfully learned 3D object geometry from uncalibrated videos without manual supervision.

Conclusions:

• The proposed unsupervised method offers a powerful and flexible approach to learning 3D object geometry.
• Probabilistic deep learning enhances the accuracy and reliability of 3D reconstruction from visual data.
• This work advances the field of computer vision by enabling detailed 3D understanding from readily available video content.