GS2Physics: Semantic-Region-Aware Gaussian Splatting for Physical Property Prediction

Area of Science:

• Computer Vision
• Computer Graphics
• Physics Simulation

Background:

• Predicting physical properties like stiffness and density for 3D assets is crucial for realistic virtual reality (VR) interactions.
• Current methods often rely on manual property assignment, leading to inefficiencies and inaccuracies.
• Existing approaches struggle with semantic segmentation consistency and feature alignment in 3D data.

Purpose of the Study:

• To develop a novel framework, GS2Physics, for accurate physical property prediction and semantic segmentation of 3D assets.
• To improve the consistency and accuracy of physical property prediction in 3D reconstructed models.
• To enhance the realism of 3D interactions by providing accurate physical properties.

Main Methods:

• Utilized 3D Gaussian Splatting as the core representation for embedding semantic-region-aware features.
• Developed GS2Physics to directly integrate semantic and physical information into the Gaussian Splatting representation.
• Introduced PhysSeg-15, a new dataset for evaluating physical property segmentation.

Main Results:

• Achieved state-of-the-art performance on the ABO-500 mass prediction benchmark.
• Demonstrated significantly improved segmentation accuracy compared to existing methods on the PhysSeg-15 dataset.
• Showcased more consistent material predictions and accurate physical property estimation across different object regions.

Conclusions:

• GS2Physics offers a robust solution for region-consistent and accurate physical property prediction in 3D assets.
• The framework enhances semantic segmentation, leading to better alignment of 3D features.
• Predicted physical properties from GS2Physics enable more realistic object motion in 3D interaction tasks.