Glissando-Net: Deep Single View Category Level Pose Estimation and 3D Reconstruction.

Area of Science:

• Computer Vision
• Deep Learning
• 3D Reconstruction

Background:

• Existing methods often focus on either object pose estimation or 3D shape reconstruction, not both simultaneously.
• Category-level 3D understanding from single images remains a challenge.

Purpose of the Study:

• To introduce Glissando-Net, a deep learning model for simultaneous pose estimation and category-level 3D shape reconstruction from single RGB images.
• To improve accuracy by enabling effective 2D-3D interaction and leveraging 3D point cloud information during training.

Main Methods:

• Glissando-Net utilizes two jointly trained auto-encoders: one for RGB images and one for point clouds.
• Key design choices include augmenting point cloud features with image decoder features and predicting both shape and pose in the decoder stage.
• The model is inspired by codeSLAM but adapted for object-centric pose and shape reconstruction without code optimization.

Main Results:

• Extensive experiments and ablation studies demonstrate the efficacy of Glissando-Net.
• The proposed method achieves state-of-the-art performance in simultaneous pose estimation and 3D shape reconstruction.
• Glissando-Net effectively integrates 2D image data with 3D shape and pose information.

Conclusions:

• Glissando-Net offers a significant advancement in single-image 3D object understanding.
• The model's architecture facilitates accurate and simultaneous prediction of object pose and 3D shape.
• This work paves the way for more robust 3D computer vision applications.