Robotic Characterization of Markerless Hand-Tracking on Meta Quest Pro and Quest 3 Virtual Reality Headsets

Area of Science:

• Human-Computer Interaction
• Virtual Reality
• Robotics

Background:

• Markerless hand-tracking enhances virtual and mixed reality (VR/MR) naturalness and immersion.
• Limited standardized evaluations exist for commercial head-mounted display (HMD) hand-tracking performance.
• Accurate hand-tracking is crucial for effective VR/MR interactions.

Purpose of the Study:

• To propose and validate a robotic methodology for evaluating markerless hand-tracking performance.
• To establish a standardized framework for characterizing hand-tracking accuracy, jitter, and latency.
• To assess the performance of Meta Quest Pro and Quest 3 mixed reality devices.

Main Methods:

• Developed a robotic manipulator system for precise measurement of hand-tracking metrics.
• Quantified positional accuracy, jitter, and latency of markerless hand-tracking systems.
• Applied the methodology to Meta Quest Pro and Quest 3, analyzing factors like hand proximity, rotation, and tracking points (knuckle vs. fingertip).

Main Results:

• Knuckle tracking yielded lower error and jitter compared to fingertip tracking for both headsets.
• Quest Pro showed superior positional accuracy (1.22 cm error) over Quest 3 (1.73 cm) in optimal conditions.
• Quest 3 demonstrated lower jitter (1.11 cm) than Quest Pro (1.77 cm); latency was highly variable for both devices (14.4-229.2 ms).

Conclusions:

• The proposed robotic methodology provides systematic and repeatable performance measurements for markerless hand-tracking.
• Hand-tracking performance is significantly influenced by tracking point selection, hand orientation, and proximity.
• This framework enables objective comparison and characterization of hand-tracking technologies in commercial VR/MR headsets.