Semantic Image Inpainting with Multi-Stage Feature Reasoning Generative Adversarial Network.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Deep Learning

Background:

• Existing image inpainting methods struggle with large missing regions due to insufficient context.
• Restoring irregular holes in images remains a significant challenge in computer vision.

Purpose of the Study:

• To propose a novel Multi-stage Feature Reasoning Generative Adversarial Network (MFR-GAN) for effective image inpainting.
• To address the limitations of current methods in handling large, irregular missing areas.

Main Methods:

• Utilized dynamic partial convolution to adaptively adjust restoration proportions and enhance pixel correlations.
• Designed a novel decoder with point-wise normalization and skip connections to manage feature statistics and prevent information loss.
• Implemented a hybrid weighted merging method (hard and soft weight maps) to combine feature maps and mitigate gradient vanishing.

Main Results:

• The proposed MFR-GAN demonstrated superior performance in restoring large missing regions compared to existing techniques.
• Achieved a Peak Signal-to-Noise Ratio (PSNR) improvement ranging from 0.3 dB to 1.2 dB on benchmark datasets (CelebA, Places2, Paris StreetView).

Conclusions:

• The MFR-GAN effectively addresses the challenge of inpainting large irregular holes by incorporating multi-stage feature reasoning.
• The novel architectural components and merging strategy contribute to significant performance gains in image restoration tasks.