A facial structure sampling contrastive learning method for sketch facial synthesis.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Contrastive learning is effective for image translation tasks like sketch face synthesis.
• Traditional contrastive learning methods struggle with random sampling and sample imbalance, leading to poor local details in synthesized sketches.

Purpose of the Study:

• To propose a novel facial structure sampling contrastive learning method for improved sketch face synthesis.
• To address the challenges of poor local detail generation caused by random sampling and sample imbalance in existing methods.

Main Methods:

• Developed a region-constrained sampling module using a dual-branch attention mechanism to segment photos based on facial structure distribution.
• Implemented a dynamic sampling strategy that adjusts sampling frequency according to feature density to mitigate sample imbalance.
• Incorporated input photo masks to reduce background influence and enhance contour delineation.
• Introduced pixel-wise and perceptual losses to further improve synthesized sketch quality.

Main Results:

• The proposed method generates high-quality sketch face images.
• Experimental results on the CUFS dataset show superior performance compared to state-of-the-art methods in both subjective and objective evaluations.
• The facial structure sampling and dynamic adjustment strategies effectively improve the representation of local details.

Conclusions:

• The Facial Structure Sampling Contrastive Learning Method significantly enhances sketch face synthesis quality.
• The proposed techniques effectively address limitations of traditional contrastive learning in this domain.
• This work offers a promising direction for generating more accurate and detailed facial sketches from photographs.