Content-Adaptive Sketch Portrait Generation by Decompositional Representation Learning.

Area of Science:

• Computer Vision and Image Processing
• Machine Learning for Graphics

Background:

• Existing sketch portrait generation methods often produce artifacts and lose fine details, particularly in areas like hair and glasses.
• Current systems struggle with generalization, requiring extensive example datasets or complex feature tuning.

Purpose of the Study:

• To develop a novel representation learning framework for end-to-end photo-to-sketch conversion.
• To address limitations in detail preservation and generalization in current sketch generation techniques.

Main Methods:

• Utilized a pre-trained Convolutional Neural Network (CNN) to decompose input photos into structural and textural components.
• Employed a branched fully CNN for learning separate structural and textural representations.
• Incorporated a sorted matching mean square error metric in the loss function for precise texture pattern measurement.

Main Results:

• The proposed framework successfully generates structural and textural representations for sketch rendering via probabilistic fusion.
• Achieved superior performance over example-based synthesis algorithms in both perceptual and objective evaluations.
• Demonstrated enhanced generalization capabilities across datasets without requiring additional training.

Conclusions:

• The novel representation learning framework effectively generates vivid, detail-preserving sketch portraits.
• The structure-texture decomposition approach significantly improves generalization ability compared to existing methods.
• This technique offers a robust solution for applications requiring high-quality, automated sketch portrait generation.