Deep Correlated Holistic Metric Learning for Sketch-Based 3D Shape Retrieval.

Area of Science:

• Computer Vision
• Machine Learning
• 3D Shape Analysis

Background:

• Effective 3D model retrieval is challenging, especially without a direct 3D query model.
• Sketch-based retrieval offers a promising, efficient alternative due to widespread electronic devices.
• A significant modality gap exists between sketches and 3D shapes, hindering direct comparison.

Purpose of the Study:

• To propose a novel deep learning method, Deep Correlated Holistic Metric Learning (DCHML), for sketch-based 3D shape retrieval.
• To mitigate the modality gap between sketch and 3D shape domains.
• To enhance the accuracy and efficiency of 3D model retrieval using sketches.

Main Methods:

• Developed DCHML, a method training two distinct deep neural networks jointly for sketch and 3D shape domains.
• Implemented a novel loss function combining discriminative and correlation losses to map features into a shared space.
• Applied loss functions at both hidden and output layers to refine feature representations.

Main Results:

• The DCHML method successfully mapped sketch and 3D shape features into a common space, reducing the modality gap.
• Experimental results on three benchmarks (3D Shape Retrieval Contest 2013, 2014, 2016) demonstrated superior performance.
• The proposed method outperformed existing state-of-the-art techniques in sketch-based 3D shape retrieval.

Conclusions:

• DCHML effectively addresses the modality gap in sketch-based 3D shape retrieval.
• The joint training of deep networks and novel loss functions leads to significant performance improvements.
• This approach offers a robust solution for efficient and accurate 3D model retrieval using simple sketch queries.