Enhancing Social Experiences in Immersive Virtual Reality with Artificial Facial Mimicry

Area of Science:

• Human-Computer Interaction
• Virtual Reality
• Social Computing

Background:

• Social Virtual Reality (SVR) platforms are expanding due to accessible hardware and Metaverse interest.
• SVR enables immersive interactions via avatars, offering potential beyond real-world limitations.
• Artificial Facial Mimicry (AFM) can augment SVR experiences by enhancing or modifying avatar emotional expressions.

Purpose of the Study:

• To investigate the potential benefits of AFM in avatar-mediated human-to-human communication within SVR.
• To assess if AFM can improve social dynamics and overcome limitations of current facial tracking technology.
• To evaluate the impact of AFM on interpersonal connection and avatar behavioral naturalness.

Main Methods:

• Developed an automated system providing AFM, nodding, and eye contact for consumer VR devices with facial tracking.
• Deployed the system in a test-bench immersive SVR application.
• Conducted a between-dyads user study comparing AFM-enabled communication against a baseline condition.

Main Results:

• AFM significantly improved subjective measures of interpersonal closeness and social attraction.
• Participants reported enhanced interpersonal trust and social presence with AFM enabled.
• The study found that AFM contributed to a greater sense of naturalness in avatar interactions.

Conclusions:

• AFM positively impacts key aspects of social interaction in SVR.
• The findings highlight the potential of AFM to enrich user experiences across diverse SVR applications.
• AFM offers a promising avenue for advancing avatar-mediated communication and social presence.