Look, Compare and Draw: Differential Query Transformer for Automatic Oil Painting

Area of Science:

• Computer Vision
• Artificial Intelligence
• Digital Art

Background:

• Automatic oil painting models often produce repetitive and uninspired brushstrokes.
• Achieving aesthetic and dynamic brushstrokes is a key challenge in neural style transfer and image synthesis.

Purpose of the Study:

• To develop a novel approach for automatic oil painting that generates dynamic and expressive brushstrokes.
• To mitigate the issue of duplicate and common-place strokes in synthesized oil paintings.

Main Methods:

• Incorporation of differential image analysis inspired by the human painting process.
• Proposal of the Differential Query Transformer (DQ-Transformer) architecture.
• Leveraging differentially derived image representations with positional encoding for stroke prediction.
• Integration of adversarial training to enhance stroke prediction accuracy and realism.

Main Results:

• The DQ-Transformer effectively concentrates on the incremental impact of successive brushstrokes.
• The model demonstrates heightened sensitivity to local details, leading to refined stroke generation.
• Adversarial training improved the realism and fidelity of synthesized paintings.
• Extensive evaluations show the DQ-Transformer surpasses existing methods in visual realism and artistic authenticity.

Conclusions:

• The proposed DQ-Transformer offers a significant advancement in automatic oil painting.
• The method achieves superior aesthetic outcomes with fewer strokes compared to prior approaches.
• Differential image analysis is a promising technique for improving the expressiveness of AI-generated art.