Hyperbolic Hierarchical 3D Scene Representations for Open-Vocabulary 3D Scene Graph Generation

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Open-vocabulary 3D scene graph generation aims to identify objects and relationships beyond predefined labels, offering greater flexibility than closed-set methods.
• Current open-vocabulary approaches often overlook higher-level scene structures, leading to performance limitations due to overfitting on individual features.

Purpose of the Study:

• To enhance open-vocabulary 3D scene graph generation by incorporating hierarchical 3D scene representations.
• To leverage hyperbolic geometry for learning robust, hierarchical scene structures (scene-region-instance) for improved 3D scene understanding.

Main Methods:

• A novel hyperbolic learning approach is proposed to embed 3D scene hierarchies into a hyperbolic representation space.
• The method optimizes embeddings in a bottom-up manner, deriving higher-level scene representations from child nodes.
• An outlier-robust adaptive distance metric is introduced to mitigate the impact of erroneous embeddings on hierarchical reasoning.

Main Results:

• The proposed method effectively generates 3D scene graphs in closed-set, open-vocabulary, and zero-shot settings.
• Experiments on 3DSSG and ScanNet datasets validate the approach's superior performance compared to existing methods.
• The integration of hierarchical representations provides crucial structural and semantic guidance for scene graph generation.

Conclusions:

• Hyperbolic learning offers a powerful framework for capturing hierarchical structures in 3D scenes, significantly advancing open-vocabulary scene graph generation.
• The developed method demonstrates improved generalization and robustness, addressing key limitations of prior work.
• The approach shows promise for more practical and less annotation-dependent 3D scene understanding applications.