SAC-GAN: Structure-Aware Image Composition.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Image composition is challenging, often resulting in unrealistic placements due to a lack of semantic and structural coherence.
• Existing methods may prioritize pixel accuracy over the overall plausibility of the composite image.

Purpose of the Study:

• To introduce an end-to-end learning framework for plausible image-to-image composition.
• To enhance semantic and structural coherence in composite images.

Main Methods:

• Developed a novel framework utilizing structure-aware features for input and output.
• Employed self-supervised learning for ground truth establishment via object cropping.
• Designed a network that generates 2D spatial affine transforms for object patch integration.
• Utilized a differentiable spatial transformer network for patch transformation.
• Implemented adversarial training with affine transform and layout discriminators.

Main Results:

• The proposed framework, SAC-GAN, demonstrates superior performance in image composition tasks.
• Evaluated SAC-GAN for quality, composability, and generalizability across various scenarios.
• Comparisons show superiority over state-of-the-art methods like Instance Insertion, ST-GAN, CompGAN, and PlaceNet.

Conclusions:

• The developed framework achieves plausible image composition by prioritizing semantic and structural coherence.
• SAC-GAN offers a robust and generalizable solution for integrating objects into scene images.