AnchorCrafter: Animate Cyber-Anchors Selling Your Products via Human-Object Interacting Video Generation

Area of Science:

• Computer Vision
• Artificial Intelligence
• Multimedia Processing

Background:

• Generating anchor-style product promotion videos is crucial for e-commerce and advertising but remains challenging.
• Existing pose-guided human video generation methods struggle with integrating human-object interactions (HOI).

Purpose of the Study:

• To introduce AnchorCrafter, a novel diffusion-based system for generating high-fidelity 2D videos of target humans interacting with customized objects.
• To address the core issue of integrating HOI into pose-guided video generation for improved product promotion videos.

Main Methods:

• Developed AnchorCrafter, a diffusion-based system incorporating HOI-appearance perception and HOI-motion injection.
• HOI-appearance perception enhances object recognition and disentangles human-object appearance.
• HOI-motion injection facilitates complex interactions by managing object trajectory and inter-occlusion.

Main Results:

• AnchorCrafter improved object appearance preservation by 7.5% compared to state-of-the-art methods.
• The system achieved superior video quality and maintained human motion consistency.
• Demonstrated high visual fidelity and controllable human-object interactions in generated videos.

Conclusions:

• AnchorCrafter effectively generates high-quality product promotion videos with controllable human-object interactions.
• The proposed HOI integration methods significantly advance pose-guided human video generation.
• The system offers promising applications in e-commerce, advertising, and consumer engagement.