Binary Set Embedding for Cross-Modal Retrieval.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Cross-modal retrieval faces challenges due to the semantic gap between different data types like images and text.
• Traditional global representations often fail to adequately bridge this gap.
• Local features offer a more robust approach for handling variations within and between classes.

Purpose of the Study:

• To propose a novel unsupervised binary coding algorithm, Binary Set Embedding (BSE).
• To obtain meaningful hash codes for local image features and text words.
• To effectively map samples into a common Hamming space for improved cross-modal retrieval.

Main Methods:

• Developed Binary Set Embedding (BSE), an unsupervised binary coding algorithm.
• Utilized local features from images and word vectors from human language.
• Employed a recursive orthogonalization procedure to reduce code redundancy.

Main Results:

• BSE maps images and text into a common Hamming space using sets of local feature descriptors.
• The algorithm explores relationships among local features at both feature and domain levels.
• Demonstrated superior performance compared to state-of-the-art cross-modal hashing methods.

Conclusions:

• BSE effectively addresses the semantic gap in cross-modal retrieval.
• The proposed method offers a robust and efficient solution for hashing local features.
• Experimental results confirm the superiority of BSE for image and text queries.