CreativeSynth: Cross-Art-Attention for Artistic Image Synthesis With Multimodal Diffusion

Area of Science:

• Computer Vision
• Artificial Intelligence
• Digital Art

Background:

• Image style transfer methods often fail to capture full artistic properties like layout and semantics, leading to synthetic outputs.
• Text-to-image models struggle to convey unique painting details and maintain aesthetic coherence when modifying specific image areas.

Purpose of the Study:

• To propose a novel framework, CreativeSynth, for integrating multimodal semantic information into artwork synthesis.
• To overcome limitations of existing methods by guiding synthesis within the art domain, preserving aesthetic harmony and semantic fusion.

Main Methods:

• Developed CreativeSynth, a multi-task unified framework based on diffusion models capable of coordinating multimodal inputs.
• Employed customized attention mechanisms and Cross-Art-Attention to integrate real-world semantic content into artworks.

Main Results:

• CreativeSynth successfully integrates real-world semantic content into diverse art categories, maintaining aesthetic quality.
• The method demonstrates effective semantic fusion and aesthetic maintenance, bridging the gap between generative models and artistic expression.

Conclusions:

• CreativeSynth offers an innovative approach to artwork generation by leveraging multimodal semantics for guided synthesis.
• The framework simplifies guidance conditions and reduces disruption to artistic harmony, advancing generative art capabilities.