Cross-Modal 3D Shape Retrieval via Heterogeneous Dynamic Graph Representation.

Area of Science:

• Computer Vision
• Artificial Intelligence
• 3D Data Analysis

Background:

• Cross-modal 3D shape retrieval is vital for comparing 3D models across different data types.
• Existing methods struggle with single-modal limitations and the inherent differences (modality gap) between 3D data formats.

Purpose of the Study:

• To develop a novel network, Heterogeneous Dynamic Graph Representation (HDGR), to enhance cross-modal 3D shape retrieval.
• To address performance bottlenecks and the modality gap in existing 3D vision retrieval systems.

Main Methods:

• Proposed HDGR network utilizing heterogeneous dynamic graphs to model context-dependent relations between diverse 3D objects.
• Employed Dynamic Graph Convolution (DGConv) and Dynamic Bipartite Graph Convolution (DBConv) for feature enhancement via heterogeneous dynamic relation learning.
• Integrated intra-modal, cross-modal, and self-transformed features into a unified representation for retrieval.

Main Results:

• HDGR demonstrated state-of-the-art performance in both cross-modal and intra-modal 3D shape retrieval tasks on ModelNet10, ModelNet40, and ABO datasets.
• Achieved robust cross-modal retrieval performance even with label noise on the 3D MNIST dataset.
• Established a stable, context-enhanced, and structure-aware 3D shape representation.

Conclusions:

• The proposed HDGR network effectively overcomes limitations of previous methods in cross-modal 3D shape retrieval.
• HDGR's ability to capture heterogeneous inter-object relationships and adapt to dynamic contexts proves its effectiveness and efficiency.
• The approach offers a significant advancement for 3D vision applications requiring robust cross-modal understanding.