A Semantic Talking Style Space for Speech-Driven Facial Animation.

Area of Science:

• Computer Vision
• Computer Graphics
• Machine Learning

Background:

• Speech-driven 3D facial animation requires disentangling identity, speech content, and talking style.
• Existing methods often require labeled data or lack intuitive control over facial styles.

Purpose of the Study:

• To introduce a latent talking style space with semantic meanings for speech-driven 3D facial animation.
• To enable automatic separation and intuitive control of talking styles without manual labeling.

Main Methods:

• A self-supervision paradigm was used to learn a latent style space from 3D speech facial animations.
• A two-stage approach involving two deep neural networks was proposed to disentangle identity, speech content, and talking style.
• Training was performed on a novel dataset of 3D talking faces with diverse styles.

Main Results:

• The learned style space automatically separates high-level attributes like mouth shape and eyebrow movement.
• Different channels of the latent style code correspond to distinct semantic meanings.
• The style space allows for intuitive and flexible control of talking styles in 3D facial animation.

Conclusions:

• The proposed latent talking style space effectively disentangles semantic attributes of facial animation.
• This approach offers a powerful tool for realistic and controllable speech-driven 3D facial animation.
• The method advances the field by enabling style control without requiring explicit style labels.