Sketch2Avatar: Geometry-Guided 3D Full-Body Human Generation in 360° From Hand-Drawn Sketches

Area of Science:

• Computer Vision
• Computer Graphics
• Artificial Intelligence

Background:

• Generating 360° full-body humans is crucial for digital entertainment, education, and art.
• Existing methods lack detailed control, often relying on coarse pose information.
• Sketch-based generation offers precise control but hasn't been explored for full-body humans.

Purpose of the Study:

• To introduce Sketch2Avatar, the first generative model for 3D full-body human creation from sketches.
• To enable precise control and sketch-alignment in 360° human generation.
• To bridge the gap in current sketch-to-3D human synthesis.

Main Methods:

• Developed a novel generative model, Sketch2Avatar, for sketch-guided 3D human representation.
• Employed a transformer-based generator with spatial feature guidance and latent modulation.
• Utilized body-aware neural rendering incorporating 3D human body priors from sketches.

Main Results:

• Successfully synthesized sketch-aligned and 360°-consistent full-body human images from hand-drawn sketches.
• Demonstrated high-quality 3D representations with precise sketch-human alignment.
• Validated the model's effectiveness through experimental results and ablation studies.

Conclusions:

• Sketch2Avatar effectively translates hand-drawn sketches into photo-realistic 360° full-body humans.
• The model offers precise control over human generation, overcoming limitations of previous methods.
• The approach advances the field of 3D human content creation from expressive 2D inputs.