Look2React: Making VR NPCs Come Alive with Dynamic Vision-Guided Reactions.

Area of Science:

• Human-Computer Interaction
• Virtual Reality
• Artificial Intelligence

Background:

• Virtual reality (VR) games promise enhanced player control via body language.
• Current non-player character (NPC) systems often lack convincing responses to player pose, intent, and context, hindering immersion.

Purpose of the Study:

• To develop and evaluate Look2React, an interaction system enabling NPCs to dynamically react to player interactions in VR.
• To improve NPC believability and player engagement by incorporating vision-based reasoning for response selection.

Main Methods:

• Developed Look2React, a system using a vision-based reasoning model for NPC pose and text response selection.
• Implemented a gaze and proximity-based detection system for intuitive and resource-efficient interaction triggering.
• Conducted a user study with 20 participants comparing Look2React NPCs against contemporary NPCs in an RPG setting.

Main Results:

• Look2React significantly increased player engagement and frequency of NPC interactions.
• Participants reported higher satisfaction, enhanced social presence, and perceived NPCs as more human-like.
• Dynamic NPC reactions were found to add depth and personality to characters.

Conclusions:

• Look2React effectively enhances immersion and social presence in VR games by making NPCs more responsive and believable.
• The system demonstrates a viable approach to creating more engaging and human-like NPC interactions in virtual environments.
• Future research can explore further integration of AI for more nuanced NPC behaviors.