Spatial Steerability of GANs via Self-Supervision from Discriminator

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Generative models achieve photorealistic image synthesis.
• Controlling image generation via latent space manipulation is an active research area.
• Existing methods often require annotations and focus on global attributes.

Purpose of the Study:

• To develop a self-supervised approach for enhancing spatial steerability in GANs.
• To enable intuitive image editing without requiring additional human annotations or searching for specific latent directions.
• To improve control over image generation for customized outputs.

Main Methods:

• Introduced randomly sampled Gaussian heatmaps as spatial inductive bias in intermediate GAN layers.
• Employed a self-supervised learning strategy to align heatmaps with the GAN discriminator's attention during training.
• Integrated the DragGAN framework for fine-grained, coarse-to-fine image manipulation.

Main Results:

• Demonstrated effective spatial editing capabilities across diverse image domains: human faces, animal faces, outdoor scenes, and complex indoor scenes.
• Achieved intuitive user interaction for adjusting scene layout, object placement, and object removal via heatmaps.
• Showcased improvements in overall image synthesis quality alongside enhanced spatial control.

Conclusions:

• The proposed self-supervised method significantly enhances spatial steerability in GANs.
• This approach offers an annotation-free and intuitive way to control image generation, applicable to various scene types.
• The integration with DragGAN allows for efficient and detailed image editing, advancing generative model customization.