Complementary, Heterogeneous and Adversarial Networks for Image-to-Image Translation

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Image-to-image translation is crucial for domain transfer.
• Conditional Generative Adversarial Networks (GANs) are widely used.
• Existing multi-generator GANs use homogeneous architectures.

Purpose of the Study:

• To explore the benefits of heterogeneous generators in image-to-image translation.
• To improve the quality and realism of translated images.
• To introduce a novel approach for combining diverse generator architectures.

Main Methods:

• Developed two generators: a deep U-Net for large perception and spatial locality, and a shallow residual network for fine details and textures.
• Implemented a gated fusion network to automatically combine outputs from heterogeneous generators.
• Proposed a multi-layer discriminator integrating multi-level and multi-scale features.

Main Results:

• Demonstrated significant improvements in image quality across various image-to-image translation tasks.
• Qualitative and quantitative evaluations confirmed the effectiveness of the proposed method.
• The heterogeneous approach yielded more realistic and detailed transferred images.

Conclusions:

• Heterogeneous generators are complementary and enhance image generation.
• The gated fusion network and multi-layer discriminator effectively integrate diverse generator strengths.
• The proposed method advances the state-of-the-art in image-to-image translation.