Image Generation from Text Using StackGAN with Improved Conditional Consistency Regularization.

Area of Science:

• Multimodal learning
• Deep learning for image generation
• Generative Adversarial Networks (GANs)

Background:

• Image generation from natural language is a rapidly advancing field.
• Stacked Generative Adversarial Networks (StackGAN) can produce high-resolution images but suffer from unintelligible outputs and mode collapse.
• Existing models like AttnGAN also show limitations in text-to-image synthesis.

Purpose of the Study:

• To address the limitations of StackGAN, specifically unintelligible image generation and mode collapse.
• To enhance the fidelity and diversity of images generated from text descriptions.
• To improve the performance of text-to-image models in conditional generation tasks.

Main Methods:

• Incorporation of Improved Consistency Regularization (ICCR), a novel technique, into the StackGAN architecture.
• ICCR stabilizes adversarial learning by matching semantic information before and after data augmentation, suppressing mode collapse.
• Modification of ICR to ICCR to eliminate generator-induced negative impacts and enhance conditional generation.

Main Results:

• StackGAN with ICCR achieved a 16% higher Inception Score than original StackGAN on the CUB dataset.
• The proposed model outperformed AttnGAN by 4% on the Inception Score.
• StackGAN with ICCR completely eliminated mode collapse (0% probability compared to 20% in original StackGAN) and received higher user ratings.

Conclusions:

• The proposed ICCR method significantly enhances StackGAN's ability to generate high-quality, relevant images from text.
• ICCR is more effective than ICR for conditional generation tasks, improving both image quality and stability.
• The developed model represents a substantial advancement in text-to-image generation, offering improved performance and reduced mode collapse.