RealCustom++: Representing Images as Real Textual Word for Real-Time Customization

Area of Science:

• Computer Science
• Artificial Intelligence
• Image Generation

Background:

• Text-to-image customization aims to align generated images with both text descriptions and source image subjects.
• Existing methods use a pseudo-word paradigm, creating conflicts between subject similarity and text controllability.
• This pseudo-word approach leads to a dual-optimum paradox, hindering simultaneous optimization of desired image attributes.

Purpose of the Study:

• To propose RealCustom++, a novel real-word paradigm for text-to-image customization.
• To disentangle the influence of text and subject for simultaneous optimization of image generation.
• To improve subject similarity, text controllability, and overall image quality in customized image generation.

Main Methods:

• Introduced RealCustom++, a real-word paradigm utilizing non-conflicting real words for subject representation.
• Developed a train-inference decoupled framework for general visual-textual alignment learning and specialized inference.
• Employed a dual-branch architecture during inference: Guidance Branch for subject mask generation and Generation Branch for region-specific customization.

Main Results:

• RealCustom++ significantly improved controllability by 7.48%, similarity by 3.04%, and quality by 76.43%.
• Achieved further improvements in controllability (4.6%) and multi-subject similarity (6.34%) for multi-subject customization tasks.
• Demonstrated superior performance over existing methods in text-to-image customization.

Conclusions:

• The proposed real-word paradigm effectively addresses the limitations of pseudo-word methods in text-to-image customization.
• RealCustom++ enables simultaneous optimization of subject similarity, text controllability, and image quality.
• The method shows strong potential for advanced image generation and multi-subject customization applications.