Unsupervised Temporal Correspondence Learning for Unified Video Object Removal.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Video object removal is crucial for editing and content creation.
• Current methods often separate mask tracking and video completion, leading to complex pipelines.
• Existing approaches require separate, tailored designs for each sub-task.

Purpose of the Study:

• To propose a unified framework for video object removal, integrating mask tracking and completion.
• To leverage inherent pixel-level temporal correspondences between these sub-tasks.
• To develop an end-to-end, unsupervised learning approach for video object removal.

Main Methods:

• A single network is proposed to infer temporal correspondences across multiple frames.
• The network links mask tracking (valid-valid pixel pairs) and video completion (valid-hole pixel pairs).
• The unified framework allows for end-to-end, unsupervised learning without annotations.

Main Results:

• The proposed method generates visually pleasing results in video object removal.
• The unified approach performs favorably against existing separate solutions.
• Demonstrated effectiveness in realistic test cases.

Conclusions:

• Unified video object removal offers a more integrated and potentially efficient solution.
• Leveraging temporal correspondences within a single framework enhances performance.
• The unsupervised, end-to-end approach simplifies deployment and broadens applicability.