IS-Diff: Improving Diffusion-Based Inpainting with Better Initial Seed

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Diffusion models excel at free-form image inpainting, producing realistic results.
• Existing methods struggle with semantic coherence due to random initial noise, causing biased inpainting.
• Inconsistent appearance between inpainted and original regions is a common issue.

Purpose of the Study:

• To address semantic and appearance inconsistencies in diffusion model-based inpainting.
• To propose a training-free approach for improved image inpainting results.
• To enhance contextual coherence in inpainted image regions.

Main Methods:

• Introduced the Initial Seed refined Diffusion Model (IS-Diff).
• Employed distributional harmonious seeds sampled from unmasked regions for initialization.
• Incorporated a dynamic selective refinement mechanism for adaptive prior adjustment.

Main Results:

• IS-Diff demonstrated superior performance in standard and large-mask inpainting tasks.
• Achieved improved semantic coherence and consistent appearance compared to state-of-the-art methods.
• Validated effectiveness on CelebA-HQ, ImageNet, and Places2 datasets.

Conclusions:

• IS-Diff effectively resolves semantic and appearance mismatches in diffusion model inpainting.
• The training-free approach offers a practical solution for high-quality image completion.
• Distributional harmonious seeds significantly enhance inpainting quality and reliability.