Multimodal cross-system virtual reality (VR) ball throwing dataset for VR biometrics

Area of Science:

• Biometrics
• Virtual Reality (VR)
• Human-Computer Interaction (HCI)

Background:

• Virtual Reality (VR) systems offer immersive experiences but present challenges in consistent biometric data collection across different hardware.
• Existing VR biometric datasets often lack multi-system synchronization and external user-view data, limiting comprehensive analysis.
• Understanding user movement patterns in VR is crucial for security, personalization, and performance optimization.

Purpose of the Study:

• To introduce a novel multimodal, cross-system dataset for virtual reality (VR) biometrics.
• To provide synchronized data from multiple VR systems (Meta Quest, HTC Vive, HTC Vive Cosmos) and external video capture.
• To enable research into cross-system VR biometrics, body keypoint analysis, and motion trajectory prediction.

Main Methods:

• Collected data from 41 participants performing a ball-throwing task across multiple VR systems.
• Utilized Meta Quest, HTC Vive, and HTC Vive Cosmos, capturing headset and controller data (position, orientation).
• Integrated external video (GoPro Hero 7) to capture full-body movements, generating COCO body keypoints (OpenPose/MMPose) and synchronized with VR data.

Main Results:

• The dataset is the first known to combine multi-VR system data with external user video for biometrics.
• Includes synchronized VR system data (NumPy), external video (MP4), and body keypoints (JSON).
• Provides comprehensive participant demographics and temporal data for advanced analysis.

Conclusions:

• The dataset supports research in cross-system VR biometrics, leveraging both internal VR tracking and external body keypoints.
• Enables the study of demographic impacts on VR biometrics and short-term movement evolution.
• Facilitates the development of 2D-to-3D and 3D-to-2D motion trajectory prediction models.