High-resolution image inpainting using a probabilistic framework for diverse images with large arbitrary masks

Area of Science:

• Computer Vision
• Machine Learning
• Image Processing

Background:

• Supervised machine learning for image inpainting requires extensive retraining for minor input changes, limiting efficiency.
• Generative Adversarial Networks (GANs) show promise but struggle with photorealistic high-resolution image inpainting, especially with large masks, often resulting in deformed structures and blurry textures.

Purpose of the Study:

• To develop a novel unsupervised probabilistic model for high-resolution image inpainting that addresses limitations of current state-of-the-art methods.
• To improve the quality, realism, and structural integrity of inpainted images, particularly for large arbitrary masks.

Main Methods:

• A novel probabilistic model is proposed, leveraging picture priors derived from StyleGAN3.
• Priors are constructed using cosine similarity, mean, and intensity (computed via the improved Papoulis-Gerchberg algorithm).
• Image reconstruction employs probabilistic maximum a posteriori estimation, with optimization via variational inference and a modified Bayes-by-Backprop approach.

Main Results:

• The proposed model demonstrates superior performance compared to state-of-the-art techniques in reconstruction quality.
• Evaluated on diverse datasets (Flickr-Faces-HQ, DIV2K, brain), the model achieves high-quality inpainting results.
• The method effectively handles high-resolution images and large arbitrary masks, producing photorealistic outputs with improved textures and structures.

Conclusions:

• The novel probabilistic model offers an efficient and effective unsupervised solution for challenging high-resolution image inpainting tasks.
• The approach successfully integrates picture priors within a generative framework to enhance image reconstruction quality.
• This method advances the field of image restoration by providing robust and high-fidelity results for complex inpainting scenarios.