Emotional Voice Puppetry.

Area of Science:

• Computer Graphics
• Human-Computer Interaction
• Animation

Background:

• Facial animation is crucial for realistic virtual characters.
• Existing methods often lack emotional expressiveness or generalizability.
• Audio-driven animation requires sophisticated control over facial dynamics.

Purpose of the Study:

• To introduce emotional voice puppetry, an audio-based facial animation technique.
• To enable vivid emotional portrayal in digital characters through voice.
• To achieve generalizability across multiple characters.

Main Methods:

• Utilizing audio content to control lip motion and surrounding facial areas.
• Establishing facial dynamics based on emotion category and intensity.
• Incorporating perceptual validity and geometric considerations over purely geometric methods.
• Investigating character-specific rig parameter categorization (eye, eyebrows, nose, mouth, wrinkles) for generalization.

Main Results:

• The approach successfully animates facial expressions based on vocal input.
• Categorizing rig parameters significantly improves generalization to new characters compared to joint training.
• User studies confirm the approach's effectiveness both qualitatively and quantitatively.

Conclusions:

• Emotional voice puppetry offers a robust method for audio-driven facial animation with emotional depth.
• The technique demonstrates strong generalizability, making it suitable for diverse character applications.
• Potential applications include AR/VR, 3DUI, virtual avatars, teleconferencing, and in-game dialogue.