Face image-sketch synthesis via generative adversarial fusion

Area of Science:

• Computer Vision
• Artificial Intelligence
• Image Synthesis

Background:

• Face image-sketch synthesis is crucial for law enforcement and entertainment.
• Existing methods struggle with realistic color generation, illumination, and high complexity.
• Generating vivid, realistic color face images from sketches remains a challenge.

Purpose of the Study:

• To develop an end-to-end generative adversarial fusion model (GAF) for synthesizing color face images from sketches.
• To improve the realism and vividness of synthesized face images by effectively learning illumination distribution.
• To reduce computational complexity and memory consumption compared to existing approaches.

Main Methods:

• Proposed a novel end-to-end generative adversarial fusion model (GAF) combining two U-Net generators and a discriminator.
• Introduced a parametric tanh activation function for learning and controlling illumination highlight distribution.
• Integrated an attention mechanism into the second U-Net generator to enhance identity consistency and facial details.

Main Results:

• GAF demonstrated superior performance over existing methods in synthesized face image quality (FSIM) and face recognition accuracy (NLDA).
• The model exhibited good generalization ability on public benchmark datasets.
• Experiments validated the effectiveness of the parametric tanh activation and attention mechanism.

Conclusions:

• The proposed GAF model effectively synthesizes high-quality, realistic color face images from sketches.
• GAF offers improved illumination control, identity preservation, and computational efficiency.
• The model shows potential for applications in facial spoofing benchmarks and demonstrates robust performance.