Streamlined Facial Data Collection Based on Utterance and Emotional Data for Human-to-Avatar Reconstruction

Area of Science:

• Computer Vision
• Human-Computer Interaction
• Virtual Reality

Background:

• Existing facial data collection methods for conversational contexts are often data-intensive and overlook user experience.
• Prioritizing technical metrics over perceived realism limits the effectiveness of current avatar reconstruction techniques.

Purpose of the Study:

• To identify essential facial expression data for efficient, photorealistic avatar reconstruction.
• To evaluate a streamlined data collection method focusing on user perception and experience.

Main Methods:

• A two-phase methodology involving facial data acquisition and user evaluation.
• Phase 1: Data acquisition and reconstruction performance assessment using utterance and emotional data.
• Phase 2: User evaluation comparing utterance-only, utterance+emotional data, and extensive data conditions.

Main Results:

• Targeted utterance and emotional data achieved realism, naturalness, and telepresence comparable to extensive data.
• The streamlined approach significantly reduced training time and data requirements.
• User evaluation with 24 participants in simulated conversations validated the findings.

Conclusions:

• Efficient avatar face reconstruction is achievable with targeted facial data inputs.
• This method offers practical guidelines for real-time applications like AR/VR telepresence.
• A balance between data quantity and perceived quality is crucial for effective avatar systems.