Improved and Accelerated Text-to-Image Generation With Collect, Reflect, and Refine

Area of Science:

• Artificial Intelligence
• Computer Vision
• Machine Learning

Background:

• Enhancing generative capabilities of text-to-image (T2I) models is crucial for AI advancements.
• Existing methods often optimize for either generative quality or inference speed, not both simultaneously.
• Current inference-enhancement techniques show limited simultaneous improvement across diffusion models (DMs) and autoregressive models (ARMs).

Purpose of the Study:

• To introduce a general tuning-based inference-enhancement framework, CoRe^2, for T2I models.
• To achieve simultaneous improvements in generative quality and reduced inference overhead for both DMs and ARMs.
• To provide a unified solution addressing the limitations of prior T2I enhancement methods.

Main Methods:

• CoRe^2 employs a three-stage process: Collect, Reflect, and Refine.
• Classifier-free guidance (CFG) trajectories are collected and used to train a weak model in the Reflect stage.
• The weak model refines difficult content in early steps and generates easy content in later steps, reducing inference time.

Main Results:

• CoRe^2 demonstrates significant performance improvements on benchmarks like HPD v2, Pick-of-Pic, Drawbench, GenEval, and T2I-Compbench.
• The framework shows effectiveness across diverse T2I models including SDXL, SD3.5, FLUX, and LlamaGen.
• Integration with Z-Sampling on SD3.5 resulted in superior performance (73% and 69% win rates on PickScore and AES) with reduced inference time.

Conclusions:

• CoRe^2 is the first framework to simultaneously enhance generative quality and reduce inference overhead for DMs and ARMs.
• The proposed method offers a general and effective solution for improving T2I model performance.
• CoRe^2 represents a significant advancement in efficient and high-quality text-to-image generation.