Multimodal Mutual Information Maximization: A Novel Approach for Unsupervised Deep Cross-Modal Hashing

Area of Science:

• Machine Learning
• Computer Vision
• Information Retrieval

Background:

• Unsupervised learning of binary hash codes is crucial for efficient cross-modal retrieval.
• Existing methods often struggle to preserve both intramodal and intermodal similarities effectively.

Purpose of the Study:

• To propose a novel unsupervised method, cross-modal info-max hashing (CMIMH), for learning binary hash codes.
• To enhance the performance of cross-modal retrieval by maximizing mutual information (MI).

Main Methods:

• Leveraging advances in estimating the variational lower bound of MI.
• Maximizing MI between binary representations and input features, and between different modalities.
• Modeling binary representations using multivariate Bernoulli distributions and employing mini-batch gradient descent.

Main Results:

• Successfully learned binary representations preserving both intramodal and intermodal similarities.
• Demonstrated the importance of balancing modality gap reduction and preserving modality-private information.
• Achieved superior performance compared to state-of-the-art methods on benchmark datasets.

Conclusions:

• The proposed CMIMH method effectively addresses unsupervised learning for cross-modal retrieval.
• Balancing information preservation is key for optimal cross-modal retrieval performance.
• CMIMH offers a significant advancement in efficient and accurate cross-modal retrieval.