Toward Fine-Grained Sketch-Based 3D Shape Retrieval.

Area of Science:

• Computer Vision
• 3D Shape Analysis
• Machine Learning

Background:

• Traditional sketch-based 3D shape retrieval (SBSR) matches objects at the category level.
• Instance-level retrieval requires precise matching between a 2D sketch and specific 3D models.
• Existing datasets lack the necessary one-to-one sketch-3D correspondences for fine-grained retrieval.

Purpose of the Study:

• To introduce and address the novel problem of fine-grained sketch-based 3D shape retrieval (FG-SBSR).
• To develop methods for instance-level 3D shape retrieval using 2D sketches.
• To bridge the domain gap between 2D sketch representations and 3D shape data.

Main Methods:

• Creation of two new datasets comprising 4,680 sketch-3D pairings across two object categories.
• Development of the first cross-modal deep embedding model specifically designed for FG-SBSR.
• Implementation of a novel cross-modal view attention module to optimize 2D projections of 3D shapes.

Main Results:

• Established the feasibility of FG-SBSR through the introduction of comprehensive datasets.
• Demonstrated the effectiveness of the proposed cross-modal deep embedding model in tackling FG-SBSR challenges.
• The cross-modal view attention module successfully computes optimal 2D shape projections for retrieval.

Conclusions:

• FG-SBSR is a challenging but achievable task with the proposed datasets and model.
• The developed deep embedding model effectively addresses the domain gap and instance-level matching requirements.
• This work lays the foundation for future research in fine-grained sketch-based 3D shape retrieval.