Repurposing GANs for One-Shot Semantic Part Segmentation

While GANs have shown success in realistic image generation, the idea of using GANs for other tasks unrelated to synthesis is underexplored. Do GANs learn meaningful structural parts of objects during their attempt to reproduce those objects? And can image synthesis serve as an "upstream" representation learning task? In this work, we test these hypotheses and propose a simple and effective approach based on GANs for fundamental vision tasks: semantic part segmentation and landmark detection. With our approach, these tasks only require as few as one labeled example along with an unlabeled dataset, rather than thousands of examples. Our key idea is to leverage a trained GAN to extract a pixel-wise representation from the input image and use it as feature vectors for a segmentation network. Our experiments demonstrate that this GAN-derived representation is "readily discriminative" and produces surprisingly good results that are comparable to those from supervised baselines trained with significantly more labels. We believe this novel repurposing of GANs underlies a new class of unsupervised representation learning, which can generalize to many other tasks. More results are available at https://RepurposeGANs.github.io/.