Transferable Coupled Network for Zero-Shot Sketch-Based Image Retrieval.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Zero-Shot Sketch-Based Image Retrieval (ZS-SBIR) faces challenges in aligning heterogeneous features for unseen categories.
• Existing methods often neglect the explicit learning of feature extractors, hindering cross-domain transferability.

Purpose of the Study:

• To propose a novel Transferable Coupled Network (TCN) to enhance network transferability in ZS-SBIR.
• To develop a general criterion for multi-modal zero-shot learning by balancing common and specific knowledge extraction.
• To introduce an effective semantic metric integrating local and global constraints for improved retrieval performance.

Main Methods:

• Implemented a Transferable Coupled Network (TCN) with soft weight-sharing in convolutional layers to capture similar geometric patterns.
• Utilized coupled and independent modules for mining modality-common and modality-specific knowledge, respectively.
• Developed a unified semantic metric combining local metric learning and global semantic constraints.

Main Results:

• Achieved significant performance improvements on three large-scale datasets: Sketchy, TU-Berlin, and QuickDraw.
• Demonstrated superior retrieval accuracy compared to state-of-the-art methods, exceeding them by over 12% on Sketchy.
• Validated the effectiveness of the proposed TCN and semantic metric in improving ZS-SBIR.

Conclusions:

• The proposed Transferable Coupled Network (TCN) effectively addresses the transferability issue in ZS-SBIR.
• The novel approach significantly boosts retrieval accuracy by leveraging coupled and independent modules and an integrated semantic metric.
• The method offers a substantial advancement in cross-modal retrieval under zero-shot learning constraints.