PRIMEdit: Probability Redistribution for Instance-Aware Multi-Object Video Editing With a New Benchmark Dataset

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• AI-powered video editing simplifies text-based modifications.
• Current zero-shot methods struggle with multi-object edits, causing unintended global changes or leakage.
• Existing techniques lack datasets and metrics for evaluating multi-object video editing.

Purpose of the Study:

• To develop a zero-shot framework for instance-aware multi-object video editing.
• To address challenges of unfaithful editing and editing leakage in localized multi-object video manipulation.
• To establish new benchmarks for multi-object video editing evaluation.

Main Methods:

• Proposed Probability Redistribution for Instance-aware Multi-object Video Editing (PRIMEdit) framework.
• Introduced Instance-centric Probability Redistribution (IPR) for precise localization and faithful editing.
• Developed Disentangled Multi-instance Sampling (DMS) to prevent editing leakage.
• Created the MIVE dataset for multi-instance video editing with complexity-controlled subsets.
• Introduced the Cross-Instance Accuracy (CIA) Score for evaluating editing leakage.

Main Results:

• PRIMEdit demonstrates superior editing faithfulness and accuracy compared to state-of-the-art methods.
• The Cross-Instance Accuracy (CIA) Score shows strong correlation with human judgment (Spearman's ρ = 0.86).
• Evaluations confirm PRIMEdit's effectiveness in preventing editing leakage in complex scenes.

Conclusions:

• PRIMEdit sets a new standard for zero-shot, instance-aware multi-object video editing.
• The proposed framework and evaluation metrics significantly advance the field.
• PRIMEdit offers a robust solution for precise and localized video content manipulation.