SceneHub4D: A Dataset and Evaluation Framework for 6-DoF 4D VR Scenes

Area of Science:

• Computer Vision
• Immersive Media Technologies
• 3D Reconstruction

Background:

• Volumetric video and 6-DoF scene capture are crucial for telepresence and mixed reality.
• Existing datasets are limited in duration, scope (studio subjects), and geometric detail.
• Evaluating real-world immersive applications faces practical barriers due to custom capture and reconstruction needs, hindering reproducibility.

Purpose of the Study:

• Introduce SceneHub4D, a novel dataset and evaluation framework.
• Address limitations of current volumetric datasets for real-world immersive applications.
• Facilitate research in 3D streaming and rendering without custom hardware.

Main Methods:

• Captured long, dynamic sequences in diverse real-world indoor environments.
• Utilized synchronized multi-view RGB-D streams, calibrated camera poses, and photogrammetry/LiDAR for geometry.
• Provided multiple 3D representations (point clouds, meshes, Gaussian splats) and a software toolkit.

Main Results:

• SceneHub4D dataset features long, dynamic sequences with synchronized multi-view RGB-D data and calibrated poses.
• High-resolution background geometry reconstructed using photogrammetry and LiDAR.
• Includes multiple 3D representations and supplementary metrics for evaluation.

Conclusions:

• SceneHub4D significantly lowers practical barriers for capturing, reconstructing, and evaluating volumetric data.
• Enables researchers to study immersive 3D streaming and rendering systems more effectively.
• Serves as a foundational resource for advancing volumetric media research and applications.