Bilinear Models of Parts and Appearances in Generative Adversarial Networks

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Generative Adversarial Networks (GANs) have advanced visual editing and synthesis by leveraging semantic information in their latent spaces.
• Current GAN editing methods often lack architectural flexibility, struggle with localized control, or require supervised data like segmentation masks.
• The need for unsupervised, architecture-agnostic methods for fine-grained control in GANs is a significant challenge in the field.

Purpose of the Study:

• To develop an unsupervised, architecture-agnostic approach for discovering spatial and appearance factors in GANs.
• To enable context-aware, pixel-level local image editing without requiring manual annotations or specific GAN architectures.
• To demonstrate the efficiency and accuracy of the proposed method compared to existing state-of-the-art techniques.

Main Methods:

• Proposed an unsupervised method that jointly discovers spatial part and appearance factors from GAN feature maps.
• Utilized semi-nonnegative tensor factorization applied to feature maps to extract these semantic factors.
• Demonstrated the method's ability to generate saliency maps corresponding to discovered appearance factors without explicit labels.

Main Results:

• The developed approach successfully disentangles spatial and appearance factors in a fully unsupervised manner.
• Achieved context-aware local image editing with precise pixel-level control, applicable across diverse GAN architectures and datasets.
• Demonstrated superior efficiency in training time and significantly improved accuracy for localized control compared to prior methods.

Conclusions:

• The proposed architecture-agnostic method offers an efficient and effective solution for unsupervised, localized control in GANs.
• The discovered appearance factors serve as implicit saliency maps, localizing concepts without supervision.
• This work advances the capabilities of GANs for realistic image editing and synthesis with fine-grained, intuitive control.