SketchBodyNet++: Sketch-Based 3D Human Mesh Reconstruction via Hybrid Parametric Networks

Area of Science:

• Computer Vision
• 3D Reconstruction
• Machine Learning

Background:

• Sketches are valuable for 3D human mesh generation but existing methods struggle with their abstract nature.
• A lack of large-scale sketch-to-mesh datasets hinders development and evaluation.

Purpose of the Study:

• To develop a robust framework for 3D human mesh reconstruction from sketches.
• To introduce a new large-scale dataset for sketch-based 3D human mesh research.

Main Methods:

• A hybrid framework combining a Local Image Encoder (LIE) and Global Point Encoder (GPE) for coarse-to-fine mesh fitting.
• Utilizing a graph-based refiner (GRefiner) to integrate local pose and global shape information.
• Creation of the Sketch3DS dataset with ~10,000 sketch-mesh pairs.

Main Results:

• The proposed framework accurately reconstructs 3D human meshes from sketches.
• Achieved superior performance compared to existing methods on the Sketch3DS dataset.
• Demonstrated effective alignment between input sketches and generated 3D human meshes.

Conclusions:

• The novel hybrid framework successfully addresses the challenges of 3D human mesh reconstruction from sketches.
• The Sketch3DS dataset provides a valuable resource for advancing research in this domain.
• The method offers a significant improvement in generating accurate and well-fitted 3D human meshes from sketches.