Qffusion: Controllable Portrait Video Editing via Quadrant-Grid Attention Learning.

Area of Science:

• Computer Science
• Artificial Intelligence
• Computer Vision

Background:

• Portrait video editing is a challenging task requiring stable and high-quality results.
• Existing methods often involve complex training pipelines or additional networks, limiting their efficiency and applicability.

Purpose of the Study:

• To introduce Qffusion, a novel dual-frame-guided framework for efficient and stable portrait video editing.
• To adapt a general animation framework for video editing using modified start and end frames as references.

Main Methods:

• Developed Qffusion, a framework trained on two still reference images, adaptable for video editing.
• Proposed a Quadrant-grid Arrangement (QGA) scheme for latent re-arrangement of reference images and facial conditions.
• Utilized self-attention for fused feature learning, enabling joint modeling of appearance and temporal representations under QGA.
• Introduced a Quadrant-grid Propagation (QGP) inference strategy for stable arbitrary-length video generation via recursive frame processing.

Main Results:

• Qffusion achieves stable portrait video editing by modifying only the input format of Stable Diffusion.
• The Quadrant-grid Propagation (QGP) strategy enables stable arbitrary-length video generation.
• Extensive experiments demonstrate that Qffusion outperforms state-of-the-art techniques in portrait video editing.

Conclusions:

• Qffusion offers a stable and efficient solution for portrait video editing without requiring additional networks or complex training stages.
• The proposed QGA and QGP strategies are effective for both general animation and video editing tasks.
• Qffusion represents a significant advancement in generative AI for video manipulation.