Feedback Recorrection Semantic-Based Image Inpainting Under Semi-Supervised Learning.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Current semantic-guided image inpainting methods lack dynamic feedback mechanisms.
• Unidirectional frameworks rely on pre-trained segmentation without adaptation during inpainting.

Purpose of the Study:

• To propose an innovative image inpainting methodology with semantic segmentation feedback recorrection.
• To enhance image reconstruction quality and segmentation accuracy through synergistic interaction.
• To reduce reliance on labeled data using semi-supervised learning.

Main Methods:

• Developed a framework enabling inpainting network feedback to a semantic segmentation model.
• Implemented cross-image semantic consistency for refining segmentation predictions.
• Utilized semi-supervised learning with labeled and unlabeled datasets for improved generalization.
• Conducted experiments on CelebA-HQ and Cityscapes datasets.

Main Results:

• Achieved a 5.89% LPIPS reduction and 0.52% PSNR increase on CelebA-HQ.
• Reduced LPIPS by 6.15% and increased SSIM by 1.58% on Cityscapes.
• Demonstrated significant improvements in both segmentation accuracy and inpainting performance.
• Ablation studies confirmed the effectiveness of the feedback recorrection mechanism.

Conclusions:

• The proposed method significantly surpasses existing image inpainting techniques.
• Fostering synergistic interactions between segmentation and inpainting substantially improves performance.
• The feedback recorrection mechanism is crucial for enhanced image processing.