Image Inpainting via Correlated Multi-Resolution Feature Projection

Area of Science:

• Computer Vision
• Artificial Intelligence
• Image Processing

Background:

• Image inpainting is crucial for recovering corrupted images, with existing methods facing trade-offs between quality, complexity, and information integration.
• Current approaches often use deep or two-stage networks, leading to high computational costs or suboptimal results.
• A gap exists in methods that balance inpainting quality with efficiency and correlated local-global information.

Purpose of the Study:

• To propose a novel single-stage, multi-resolution generator architecture for efficient and high-quality image inpainting.
• To address the limitations of existing methods regarding complexity and the integration of local and global image features.
• To achieve superior image reconstruction outcomes with moderate computational complexity.

Main Methods:

• Developed a single-stage multi-resolution generator architecture.
• Introduced a multi-kernel non-local (MKNL) attention block for merging feature maps across resolutions.
• Proposed a feature projection block for effective reconstruction and a valid feature fusion block to prevent redundant merging.

Main Results:

• The proposed architecture demonstrates superior image inpainting outcomes compared to state-of-the-art methods.
• Achieved high-quality results with moderate complexity, indicated by parameter count and runtime.
• Effectiveness validated on CelebA-HQ, Karras et al. 2017, and Places2 datasets using the NVIDIA mask dataset.

Conclusions:

• The proposed single-stage multi-resolution generator offers a robust and efficient solution for image inpainting.
• The novel attention and fusion blocks effectively integrate multi-resolution features for enhanced image reconstruction.
• The method shows significant promise for applications like object removal and general image restoration.