DrawingInStyles: Portrait Image Generation and Editing With Spatially Conditioned StyleGAN

Area of Science:

• Computer Vision
• Deep Learning
• Generative Models

Background:

• Deep learning has advanced sketch-to-portrait generation.
• StyleGAN offers state-of-the-art generation but lacks sketch-friendliness due to unconditional nature.
• Preserving spatial information is crucial for sketch-based image synthesis.

Purpose of the Study:

• To develop a StyleGAN-based framework that directly incorporates spatial information for sketch-to-portrait generation.
• To enhance the control and precision of generating realistic face images from user inputs like sketches and semantic maps.
• To create an intuitive drawing interface for non-professional users to produce high-quality portraits.

Main Methods:

• Propose Spatially Conditioned StyleGAN (SC-StyleGAN) by injecting spatial constraints into the StyleGAN framework.
• Utilize sketches and semantic maps as input modalities for precise user guidance.
• Develop the DrawingInStyles interface for user-friendly portrait creation and editing.

Main Results:

• SC-StyleGAN demonstrates superior generation capabilities compared to existing methods.
• The system effectively preserves spatial information from input sketches.
• Qualitative and quantitative evaluations confirm the method's effectiveness.

Conclusions:

• SC-StyleGAN offers a robust solution for controlled sketch-to-portrait generation.
• The DrawingInStyles system provides an accessible tool for creating realistic face images.
• The approach enhances both the quality of generated portraits and user interaction experience.