AniFeats: Animate 3D Feature Meshes for Character Video Generation

Area of Science:

• Computer Vision
• Computer Graphics
• Artificial Intelligence

Background:

• Generating high-quality character animation videos is challenging.
• Existing methods use geometry guidance (skeletons, normals, depth) with diffusion models.
• Current approaches suffer from temporal inconsistencies and reduced quality due to reliance on cross-attention for geometry guidance.

Purpose of the Study:

• To present AniFeats, a novel framework for generating high-quality character animation videos.
• To improve temporal consistency and realism in character animation generation.
• To address limitations of existing methods that rely solely on cross-attention for geometry guidance.

Main Methods:

• AniFeats incorporates explicit features on 3D character meshes during video generation.
• Detailed features are extracted from a reference image and projected onto 3D feature meshes (SMPL-X).
• Rendered feature maps from animated 3D feature meshes guide the generation process, linking local image patterns to video output.

Main Results:

• AniFeats achieves significantly improved temporal consistency in character animation videos.
• The framework generates high-quality animations with enhanced realism.
• Experimental results demonstrate the effectiveness of using 3D feature meshes for guidance.

Conclusions:

• AniFeats offers a novel approach to character animation by leveraging explicit 3D mesh features.
• This method effectively strengthens temporal coherence and realism in generated videos.
• AniFeats represents a significant advancement over existing geometry-guided diffusion models for character animation.