A video, text, and speech-driven realistic 3-d virtual head for human-machine interface.

Area of Science:

• Computer graphics
• Human-computer interaction
• Artificial intelligence

Background:

• Realistic facial animation is crucial for natural human-machine interfaces.
• Existing systems often struggle with diverse input modalities and lighting variations.
• Efficient and high-fidelity facial animation remains a challenge.

Purpose of the Study:

• To propose a novel multiple input-driven realistic facial animation system using a 3-D virtual head.
• To enable interaction through independent control via video, text, and speech.
• To achieve a balance between computational efficiency and animation realism.

Main Methods:

• Utilizing a combination of parameterized and muscular models for facial animation.
• Employing an online appearance model with particle filtering for 3-D facial motion tracking from video.
• Integrating multiple measurements (pixel color, Gabor wavelet coefficients) to mitigate lighting and person dependency.
• Implementing a tri-phone model for efficient speech-synchronized viseme synthesis, reducing computational load.

Main Results:

• The system successfully tracks 3-D facial motion from video inputs.
• It demonstrates robustness against variations in lighting and individual facial characteristics.
• Speech-synchronized viseme synthesis is achieved with reduced computational cost.
• Objective and subjective evaluations confirm the system's suitability for human-machine interaction.

Conclusions:

• The proposed system offers a flexible and realistic approach to facial animation for human-machine interfaces.
• Its ability to integrate multiple input modalities enhances interactive capabilities.
• The methods employed provide a good trade-off between animation quality and computational efficiency.