Time-Variant Image Inpainting via Interactive Distribution Transition Estimation

Area of Science:

• Computer Vision
• Image Restoration
• Machine Learning

Background:

• Conventional image inpainting methods struggle with time-variant images due to significant content differences and potential damage in reference images.
• Existing state-of-the-art (SOTA) reference-guided inpainting techniques fail to produce plausible results in the challenging Time-Variant Image Inpainting (TAMP) setup.
• Restoring damaged images using temporally distant references is a practical, real-world problem with no guaranteed reference quality.

Purpose of the Study:

• To address the ill-posed problem of Time-Variant Image Inpainting (TAMP).
• To propose a novel method that interactively complements time-variant images for damaged region restoration.
• To establish a new benchmark dataset for evaluating TAMP methods.

Main Methods:

• Introduction of the Interactive Distribution Transition Estimation (InDiTE) module for semantic complementation of time-variant images.
• Development of InDiTE-driven Diffusion (InDiTE-Diff), integrating InDiTE with diffusion models for latent cross-reference during sampling.
• Assembly of the TAMP-Street dataset, comprising diverse image and mask data for TAMP task evaluation.

Main Results:

• The proposed InDiTE module effectively complements semantic information between time-variant images.
• InDiTE-Diff demonstrates superior performance compared to SOTA reference-guided inpainting methods on the TAMP-Street dataset.
• Experiments validate the method's consistency and effectiveness across different TAMP settings.

Conclusions:

• The InDiTE module and InDiTE-Diff model offer a robust solution for Time-Variant Image Inpainting.
• The TAMP-Street dataset provides a valuable resource for advancing research in this novel domain.
• The proposed approach significantly outperforms existing methods in restoring damaged images using temporally distinct references.