Make-Your-Anchor+: Temporal Consistent 2D Avatar Generation via Video Diffusion Prior.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Digital Media

Background:

• Generating realistic talking-head avatars is established, but full-body motion synthesis remains a significant challenge.
• Existing methods often require extensive data or struggle with temporal coherence and identity preservation.

Purpose of the Study:

• Introduce Make-Your-Anchor+, a system for automatic anchor-style video generation with precise full-body movements.
• Enable the creation of digital avatars from minimal user input for online content creators.

Main Methods:

• Finetune a structure-guided diffusion model on a one-minute input video to render 3D mesh conditions into human appearances.
• Employ a two-stage training strategy for the diffusion model, mapping specific movements to appearances.
• Adapt frame-wise diffusion models to pretrained video diffusion weights and utilize a batch-overlapped temporal denoising module for extended video generation.
• Incorporate an identity-specific face enhancement module to improve facial region quality.

Main Results:

• The system successfully generates anchor-style videos with accurate torso and hand movements.
• Make-Your-Anchor+ demonstrates superior visual quality, temporal coherence, and identity preservation compared to state-of-the-art methods.
• The proposed temporal denoising module effectively bypasses video length constraints during inference.

Conclusions:

• Make-Your-Anchor+ offers an effective and efficient solution for creating high-quality digital avatars with full-body motion.
• The system has broad applications for online streamers, live shopping hosts, and virtual presenters.
• This work advances the state-of-the-art in generative AI for realistic human motion synthesis.