Expressive facial animation synthesis by learning speech coarticulation and expression spaces

Area of Science:

• Computer Graphics
• Machine Learning
• Animation

Background:

• Synthesizing expressive facial animation is a complex challenge in computer graphics.
• Existing methods often struggle with realistic and dynamic expression generation.

Purpose of the Study:

• To develop an automated system for synthesizing expressive facial animation.
• To leverage facial motion capture data for learning speech coarticulation and expression models.

Main Methods:

• Motion capture of facial movements during speech with expressions.
• Developing speech coarticulation models for diphones and triphones.
• Constructing a Phoneme-Independent Expression Eigenspace (PIEES) using PCA.
• Synthesizing neutral visual speech and dynamic expressions, then blending them.

Main Results:

• The system effectively learns speech coarticulation models from recorded data.
• A PIEES model was created to capture dynamic expression signals.
• The developed system successfully synthesized realistic expressive facial animations.

Conclusions:

• The proposed system demonstrates a viable approach to automated expressive facial animation synthesis.
• The integration of learned coarticulation and expression models yields realistic results.
• This method advances the field of facial animation by combining motion capture and machine learning.