Painterly Style Transfer With Learned Brush Strokes.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Digital Art

Background:

• Traditional Neural Style Transfer (NST) often uses texture patches, resulting in outputs that lack a painterly aesthetic.
• Existing NST methods struggle to replicate the nuanced application of brush strokes characteristic of real-world paintings.
• The semantic meaning and visual salience of content are not adequately captured by conventional NST approaches.

Purpose of the Study:

• To develop a novel stroke-based method for Neural Style Transfer (NST) that produces more painterly and aesthetically pleasing results.
• To analyze and model artistic brush stroke families from real-world paintings.
• To improve the emphasis of salient semantic content in stylized images by integrating language-image models.

Main Methods:

• Analysis of real-world paintings to learn distributions of stroke families based on shape (dots, lines, arcs).
• Sampling learned stroke distributions during synthesis to ensure stylization uses appropriate stroke types.
• Development of a new content loss function based on language-image models to prioritize semantically meaningful and salient image regions.

Main Results:

• The proposed stroke-based NST method generates outputs that are demonstrably more painterly compared to texture-based NST.
• Learned stroke families are effectively sampled to guide the stylization process, enhancing the artistic quality.
• The use of a language-image model-based loss function successfully emphasizes salient image content, improving output fidelity.

Conclusions:

• A stroke-based approach offers a significant improvement over texture-based methods for achieving painterly Neural Style Transfer.
• Modeling and sampling artistic stroke families are crucial for generating authentic artistic styles.
• Integrating semantic understanding via language-image models enhances the quality and focus of stylized images by emphasizing salient content.