Sketch2Avatar: Geometry-Guided 3D Full-Body Human Generation in 360° From Hand-Drawn Sketches

Generating full-body humans in 360$^{\circ }$∘ has broad applications in digital entertainment, online education and art design. Existing works primarily rely on coarse conditions such as body pose to guide the generation, lacking detailed control over the synthesized results. Regarding this limitation, sketches offer a promising alternative as an expressive condition that enables more explicit and precise control. However, current sketch-based generation methods focus on faces or common objects, how to transfer sketches into 360 $^{\circ }$∘ full-body humans remains unexplored. To bridge this gap, we propose Sketch2Avatar, the first generative model to achieve 3D full-body human generation from hand-drawn sketches. Our model is capable of synthesizing sketch-aligned and 360$^{\circ }$∘-consistent full-body human images by leveraging the geometry information extracted from sketches to guide the 3D representation generation and neural rendering. Specifically, we propose sketchguided 3D representation generation to model the 3D human and maintain the alignment between input sketches and generated humans. Our transformer-based generator incorporates spatial feature guidance and latent modulation derived from sketches to produce high-quality 3D representations. Additionally, our designed bodyaware neural rendering utilizes 3D human body priors from sketches, simplifying the learning of articulated body poses and complex body shapes. To train and evaluate our model, we construct a large-scale dataset comprising approximately 19 K 2D full-body human images and their corresponding sketches in a hand-drawn style. Experimental results demonstrate that our Sketch2Avatar can transfer hand-drawn sketches into photo-realistic 360$^{\circ }$∘ full-body human images with precise sketch-human alignment. Ablation studies further validate the effectiveness of our design choices.