Reconstructing 3D Shapes from Multiple Sketches using Direct Shape Optimization.

Area of Science:

• Computer Vision
• Computer Graphics
• 3D Modeling

Background:

• Current 3D shape reconstruction from sketches relies on neural networks mapping to voxel grids.
• Voxel-based methods face challenges with cubic complexity, limiting resolution, geometric detail, and accuracy.

Purpose of the Study:

• To propose a novel method for 3D shape reconstruction from multiple hand-drawn sketches.
• To overcome the limitations of deep learning-based voxel reconstruction methods.

Main Methods:

• Utilizes direct shape optimization (DSO) without direct deep learning for voxel generation.
• Employs a conditional generative adversarial network (CGAN) to translate sketches into predicted geometry (attenuance images).
• Minimizes a project-and-compare loss for 3D shape reconstruction, matching predicted attenuance images from multiple viewpoints.

Main Results:

• The proposed DSO method significantly outperforms state-of-the-art techniques on standard benchmarks.
• Achieves higher accuracy and geometric detail compared to existing approaches.
• Demonstrates intuitive results in an interactive application, including handling sketch inconsistencies with a progressive update approach.

Conclusions:

• Direct shape optimization offers a more effective approach for 3D shape reconstruction from multiple sketches.
• The method provides a viable alternative to computationally intensive deep learning voxel-based techniques.
• The approach enhances the quality and detail of reconstructed 3D shapes from user-provided drawings.