NoisePO: Efficient Semantic Noise Generation and Ranking for Diffusion-Based Text-to-Image Synthesis

Area of Science:

• Artificial Intelligence
• Computer Vision

Background:

• Diffusion models excel at photorealistic image generation through iterative denoising.
• Error accumulation from suboptimal noise estimation degrades semantic alignment and fidelity in multi-step denoising.

Purpose of the Study:

• Introduce NoisePO, an efficient semantic noise preference optimization framework.
• Address suboptimal inner latent representations for improved image generation quality and semantic consistency.

Main Methods:

• Employ a semantic noise preference optimization generative adversarial network (NPO-GAN) to generate semantically relevant noises.
• Utilize noise ranking models to filter and select optimal noises based on textual conditions.
• Encourage latent incorporation of semantic information from captions via generated noises.

Main Results:

• NoisePO demonstrates consistent outperformance across baseline frameworks.
• Achieved notable improvements in image quality, semantic consistency, and user-specific alignment.
• Quantifiable enhancements observed via metrics like IS, FID, and CLIP scores.

Conclusions:

• NoisePO effectively enhances synthesis quality in diffusion-based image generation.
• The framework significantly strengthens text-image alignment and latent representation fidelity.