Sketch2Human: Deep Human Generation With Disentangled Geometry and Appearance Constraints

Area of Science:

• Computer Vision
• Artificial Intelligence
• Computer Graphics

Background:

• Generating full-body human images with precise control over geometry and appearance is challenging.
• Existing methods lack explicit control or struggle with the complexity of human forms and clothing.
• Sketch-based methods show promise but face limitations in fidelity and diversity for full-body generation.

Purpose of the Study:

• To introduce Sketch2Human, the first system for controllable full-body human image generation.
• To enable explicit control over body and garment geometry via semantic sketches and appearance via reference images.
• To address the limitations of existing methods in balancing realism and sketch faithfulness.

Main Methods:

• Utilizes the latent space of StyleGAN-Human with inverted geometry and appearance latent codes.
• Employs a novel sketch encoder trained on a large synthetic dataset, directly supervised by sketches.
• Introduces a unique training scheme for disentangled geometry and appearance control using geometry-preserved and appearance-transferred data.

Main Results:

• Achieves controllable full-body human image generation with both semantic sketch and reference image guidance.
• Demonstrates superior performance compared to state-of-the-art methods in qualitative and quantitative evaluations.
• Shows effectiveness with synthetic data training while also handling hand-drawn sketches.

Conclusions:

• Sketch2Human offers a robust solution for high-fidelity, controllable full-body human image synthesis.
• The proposed method advances sketch-guided generation by enabling disentangled control over geometry and appearance.
• This work provides a significant step forward in realistic and controllable human image generation.