Progressive Learning of 3D Reconstruction Network From 2D GAN Data.

Area of Science:

• Computer Vision
• Computer Graphics
• Machine Learning

Background:

• Current 3D reconstruction methods require costly multi-view images and precise camera parameters.
• Generative Adversarial Networks (GANs) offer a cost-effective alternative using generated multi-view datasets.
• GAN-generated datasets often lack multi-view consistency and can contain distortions, degrading reconstruction quality.

Purpose of the Study:

• To develop a novel method for high-quality textured 3D model reconstruction from single images.
• To overcome the limitations of GAN-generated datasets in 3D reconstruction tasks.
• To achieve state-of-the-art results on challenging objects without expensive annotations.

Main Methods:

• A robust multi-stage learning scheme is proposed, increasing reliance on model predictions for loss calculation.
• A novel adversarial learning pipeline with online pseudo-ground truth generation is introduced for fine detail reconstruction.
• The method bridges 2D supervision from GANs to 3D reconstruction models, reducing annotation costs.

Main Results:

• The proposed method achieves state-of-the-art results on challenging 3D reconstruction tasks.
• Significant improvements are demonstrated compared to previous methods trained on both GAN-generated and real-world datasets.
• The approach effectively handles distortions and inconsistencies present in GAN-generated data.

Conclusions:

• The developed method provides a viable and cost-effective approach to 3D model reconstruction.
• It successfully bridges the gap between 2D generative models and 3D reconstruction, minimizing annotation requirements.
• This work advances the field by enabling high-quality 3D reconstruction using readily available, albeit imperfect, generated data.