Toward Realistic Face Photo-Sketch Synthesis via Composition-Aided GANs

Area of Science:

• Computer Vision
• Artificial Intelligence
• Image Synthesis

Background:

• Face photo-sketch synthesis is crucial for applications like digital entertainment and law enforcement.
• Existing methods struggle with structural realism and textural consistency, often producing blurred or deformed results.
• Synthesized images frequently lack realism due to limitations in depicting facial components accurately.

Purpose of the Study:

• To develop a novel method for generating high-fidelity face sketches and photos from corresponding inputs.
• To address the challenges of structural realism and textural consistency in face synthesis.
• To improve the overall realism and identity preservation of synthesized facial images.

Main Methods:

• Propose a Composition-aided Generative Adversarial Network (CA-GAN) utilizing paired face photos/sketches and pixelwise face labels.
• Introduce a compositional reconstruction loss to focus on hard-generated components and delicate facial structures.
• Employ a perceptual loss function for perceptual similarity and stacked CA-GANs (SCA-GANs) for defect rectification and detail enhancement.

Main Results:

• The proposed CA-GAN and SCA-GAN methods generate visually comfortable and identity-preserving face sketches and photos.
• Significantly improved Fréchet Inception Distance (FID) scores: from 36.2 to 26.2 for sketch synthesis and 60.9 to 30.5 for photo synthesis.
• Demonstrated considerable generalization ability across a wide range of challenging data.

Conclusions:

• Facial composition information effectively aids face photo-sketch synthesis.
• The proposed CA-GAN and SCA-GAN frameworks offer a significant advancement in generating realistic and identity-preserving facial images.
• The method shows strong potential for real-world applications requiring accurate face synthesis.