Sketch Augmentation-Driven Shape Retrieval Learning Framework Based on Convolutional Neural Networks.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Sketch-based shape retrieval is challenging due to data scarcity and domain differences between sketches and 3D models.
• Existing methods often struggle with generating realistic sketch data and effectively bridging the gap between 2D sketches and 3D shapes.

Purpose of the Study:

• To develop a deep learning approach for improved sketch-based 3D shape retrieval.
• To address the limitations of training data scarcity and enhance the accuracy of shape matching.

Main Methods:

• A novel sketch augmentation technique generating diverse training samples by stroke manipulation (removal, adjustment, rotation).
• A convolutional neural network (CNN) to identify optimal 2D viewpoints for 3D models, ensuring salient feature depiction.
• A cross-domain learning strategy using Siamese CNNs and a joint Bayesian measure for sketch-to-3D shape similarity assessment.

Main Results:

• The proposed sketch augmentation significantly increases the volume and diversity of training data.
• Optimized 2D renderings capture essential 3D shape characteristics effectively.
• The cross-domain learning approach successfully maximizes inter-class and minimizes intra-class similarity, outperforming state-of-the-art methods.

Conclusions:

• The integrated deep learning approach offers a robust solution for sketch-based shape retrieval.
• Novel techniques in data augmentation and cross-domain learning are key to achieving superior performance.
• This method advances the field by providing more accurate and efficient shape retrieval from user sketches.