TalkingStyle: Personalized Speech-Driven 3D Facial Animation With Style Preservation

Area of Science:

• Computer Graphics
• Artificial Intelligence
• Human-Computer Interaction

Background:

• Generating realistic 3D avatars for speech-driven facial animation is challenging.
• Current methods struggle to capture unique individual talking styles and achieve lifelike mimicry.

Purpose of the Study:

• To propose a novel method, TalkingStyle, for generating personalized talking avatars.
• To retain the unique talking style of individuals in speech-driven facial animation.
• To improve realism and lip synchronization accuracy in animated avatars.

Main Methods:

• Utilizing audio and animation samples to create personalized facial animations.
• Disentangling style codes from motion patterns using separate encoders for style, speech, and motion.
• Employing a style-conditioned transformer decoder for enhanced control over avatar styles.

Main Results:

• TalkingStyle generates facial animations that closely resemble an individual's specific talking style.
• The method achieves superior realism and lip synchronization accuracy compared to state-of-the-art methods.
• Qualitative, quantitative assessments, and user studies validate the effectiveness of TalkingStyle.

Conclusions:

• TalkingStyle offers a significant advancement in creating personalized, lifelike talking avatars.
• The approach successfully preserves individual talking styles while ensuring accurate speech synchronization.
• The public release of the source code facilitates further research in speech-driven facial animation.