Pixel2Mesh++: 3D Mesh Generation and Refinement From Multi-View Images.

Area of Science:

• Computer Vision
• Computer Graphics
• Machine Learning

Background:

• Generating 3D shapes from limited 2D images is a challenging problem in computer vision.
• Existing methods often rely on direct shape hallucination from learned priors, which can limit quality.

Purpose of the Study:

• To improve 3D shape generation quality by effectively utilizing cross-view information.
• To develop a method that iteratively refines coarse shapes for better accuracy and visual plausibility.

Main Methods:

• A graph convolution network is employed to leverage cross-view information.
• The model learns to predict iterative deformations to refine an initial coarse shape.
• Perceptual feature statistics from multiple images guide the deformation process, inspired by multi-view geometry.

Main Results:

• The proposed model generates accurate 3D shapes that are visually plausible from input views and well-aligned to arbitrary viewpoints.
• The physically driven architecture demonstrates generalization across different object categories and varying numbers of input images.
• The method shows robustness to initial mesh quality and camera pose errors.

Conclusions:

• The iterative deformation approach using cross-view information significantly enhances 3D shape generation.
• The model's generalization and robustness make it a versatile tool for 3D reconstruction tasks.
• Integration with differentiable rendering allows for further optimization during testing.