A Framework for Enabling Unpaired Multi-Modal Learning for Deep Cross-Modal Hashing Retrieval.

Area of Science:

• Information Retrieval
• Machine Learning
• Computer Vision

Background:

• Cross-Modal Hashing (CMH) methods are efficient for large-scale data retrieval.
• Current CMH methods primarily rely on paired data (e.g., image-text pairs) for training.
• Real-world applications often involve unpaired data, limiting the utility of existing CMH techniques.

Purpose of the Study:

• To provide an overview of CMH methods applicable to unpaired datasets.
• To propose a novel framework enabling pairwise-constrained CMH methods to train effectively using unpaired samples.
• To evaluate the performance of state-of-the-art CMH methods across various data pairing scenarios.

Main Methods:

• Literature review of CMH techniques for unpaired data.
• Development of a new framework to adapt pairwise-constrained CMH for unpaired training.
• Empirical evaluation of leading CMH methods on diverse datasets with varying degrees of data pairing.

Main Results:

• The proposed framework successfully enables training of pairwise-constrained CMH methods on unpaired data.
• Performance analysis across different pairing scenarios highlights the robustness and limitations of current CMH approaches.
• The study provides insights into adapting CMH for practical, real-world information retrieval tasks.

Conclusions:

• Learning from unpaired data is essential for advancing Cross-Modal Hashing in practical applications.
• The proposed framework offers a viable solution for utilizing unpaired data in CMH.
• Further research is needed to optimize CMH methods for scenarios with limited or no paired data.