SceneSketcher-v2: Fine-Grained Scene-Level Sketch-Based Image Retrieval Using Adaptive GCNs.

Area of Science:

• Computer Vision
• Machine Learning
• Image Retrieval

Background:

• Sketch-based image retrieval (SBIR) traditionally focuses on category or instance-level matching.
• Fine-grained scene-level SBIR presents unique challenges due to complex scene composition and the domain gap between sketches and images.

Purpose of the Study:

• To develop an effective method for fine-grained scene-level sketch-based image retrieval.
• To address the challenges of representing multi-modal scene information and bridging the sketch-to-image domain gap.

Main Methods:

• A novel Graph Convolutional Network (GCN) based architecture, SceneSketcher-v2, was developed.
• The GCN fuses multi-modal information from sketches and images, including object layout, relative size, and visual appearance.
• A triplet training process and end-to-end training were employed to minimize the domain gap.

Main Results:

• SceneSketcher-v2 significantly outperforms existing state-of-the-art models in scene-level SBIR.
• The GCN effectively models complex scene information and mitigates the domain gap.
• Experimental results validate the effectiveness of the proposed approach.

Conclusions:

• SceneSketcher-v2 offers a robust solution for challenging fine-grained scene-level sketch-based image retrieval.
• The GCN architecture provides a powerful framework for fusing multi-modal scene information.
• This work advances the capabilities of SBIR systems for complex scene understanding.