Exploiting Deep Generative Prior for Versatile Image Restoration and Manipulation.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Learning image priors is crucial for image restoration and manipulation.
• Existing methods like Deep Image Prior (DIP) capture low-level statistics but lack rich semantic understanding.
• There's a need for image priors that encompass color, spatial coherence, textures, and high-level concepts.

Purpose of the Study:

• To develop an effective method for exploiting image priors from generative adversarial networks (GANs) trained on natural images.
• To enable advanced image restoration and manipulation tasks using a deep generative prior (DGP).
• To overcome limitations of existing GAN inversion methods by allowing generator fine-tuning.

Main Methods:

• Utilized a generative adversarial network (GAN) trained on large-scale natural images to capture a deep generative prior (DGP).
• Relaxed the fixed-generator assumption common in GAN inversion methods.
• Implemented on-the-fly, progressive fine-tuning of the generator, regularized by discriminator-based feature distance.
• Ensured reconstructions remain within the natural image manifold.

Main Results:

• Demonstrated compelling results in restoring missing semantics (color, patch, resolution) in degraded images.
• Enabled diverse image manipulation, including random jittering, image morphing, and category transfer.
• Achieved precise and faithful reconstruction of real images due to preserved manifold properties.

Conclusions:

• Deep generative prior (DGP) offers a flexible and effective approach for image restoration and manipulation.
• The proposed method enhances semantic understanding and enables sophisticated image editing.
• This technique provides more accurate and reliable image reconstruction compared to existing methods.