Content style decoupling for multi style image generation using latent diffusion architecture

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Multi-style image generation methods struggle with content-style decoupling, high computational demands for high-resolution outputs, and structural distortions during style transfer.
• Existing models often fail to independently control content structure and style elements, limiting their effectiveness.

Purpose of the Study:

• To propose a novel Dual-Conditional Lightweight Style Diffusion Model (DCLSDM) that addresses the limitations of current multi-style image generation techniques.
• To enhance content-style decoupling for improved control in style transfer applications.
• To develop an efficient and resource-friendly solution for high-quality image generation.

Main Methods:

• Developed a Dual-Conditional Lightweight Style Diffusion Model (DCLSDM) incorporating a dual-conditional control mechanism.
• The mechanism independently manages content structure and style expression for precise control during style transfer.
• Evaluated the model on WikiArt and Summer2Winter Yosemite datasets.

Main Results:

• DCLSDM demonstrated superior performance compared to existing models, evidenced by improved SSIM, LPIPS, and FID scores.
• Achieved significant reductions in inference time, memory usage, and parameter scale.
• The model proved effective in resource-constrained environments.

Conclusions:

• The DCLSDM offers an efficient and controllable solution for multi-style image generation.
• Its lightweight design and enhanced decoupling capabilities make it suitable for various applications, including content creation and digital art production.
• The model overcomes key challenges in style transfer and high-resolution image synthesis.