Before hands disappear: Effect of early warning visual feedback method for hand tracking failures in virtual reality

Area of Science:

• Human-Computer Interaction
• Virtual Reality
• Computer Vision

Background:

• Virtual hand representation in Head-Mounted Displays (HMDs) enables immersive Virtual Reality (VR) experiences.
• Existing hand tracking algorithms often suffer from errors, negatively impacting user experience and task efficiency.

Purpose of the Study:

• To develop and evaluate a novel method for providing users with visual feedback on hand tracking quality in HMDs.
• To mitigate the negative effects of hand tracking failures on user performance and experience.

Main Methods:

• Implemented a notification modal to alert users about potential hand tracking failures.
• Identified three common hand tracking failure scenarios.
• Evaluated the system in VR object manipulation and complex assembly tasks.

Main Results:

• The early warning system reduced task completion time and hand-tracking failures by up to 83%.
• Observed significant decreases in errors and cognitive load.
• Reported improvements in system usability and overall user experience.

Conclusions:

• The proposed visual feedback system enhances the reliability and usability of hand tracking in VR HMD applications.
• This approach contributes to more robust and user-friendly VR interactions by addressing hand tracking limitations.
• The findings support the development of advanced VR systems with improved user workload management.