Enhancing Sketch-Based Image Retrieval by CNN Semantic Re-ranking.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Sketch-based image retrieval (SBIR) systems aim to find natural images matching a given sketch.
• Existing SBIR methods often struggle with accurately capturing semantic similarity between sketches and images.
• Deep learning approaches, particularly Convolutional Neural Networks (CNNs), show promise in enhancing image understanding and retrieval.

Purpose of the Study:

• To develop and evaluate a CNN-based semantic re-ranking system for improving SBIR performance.
• To leverage category information extracted by CNNs for more effective similarity measurement.
• To enhance the precision and accuracy of sketch-based image retrieval results.

Main Methods:

• A dual CNN model approach was employed, with one CNN trained for sketch classification and another for natural image classification.
• A novel category similarity measurement method was proposed to quantify semantic relationships between sketches and images.
• The system re-ranks initial retrieval results by inferring the query sketch's category and applying category similarity measures.

Main Results:

• The proposed CNN semantic re-ranking system demonstrated significant improvements in SBIR performance across various datasets.
• Experimental results showed superior precision compared to existing re-ranking algorithms and baseline systems.
• The method achieved notably higher precision in top-ten retrieval results, validating its effectiveness.

Conclusions:

• The developed CNN semantic re-ranking system effectively enhances sketch-based image retrieval by utilizing category information.
• The dual CNN model and category similarity measurement contribute to more accurate semantic understanding and retrieval.
• This approach offers a superior alternative to existing methods for improving SBIR accuracy and precision.