Twin contrastive interventional-cause hashing for unsupervised cross-modal retrieval

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Unsupervised deep cross-modal hash retrieval (UDCMH) methods often rely on similarity or contrastive loss.
• Existing methods treat contrastive learning as a plug-and-play module and use hashing solely as semantic representation.
• There's a lack of deeper analysis and integration of hashing into model construction and feature learning.

Purpose of the Study:

• To propose a novel approach for unsupervised cross-modal retrieval by integrating contrastive learning and hashing.
• To develop a framework that utilizes contrastive knowledge for group-wise causal reasoning.
• To create an intrinsically interpretable UDCMH model through interventional knowledge.

Main Methods:

• Introduced twin contrastive interventional-cause hashing (TCICH) for unsupervised cross-modal retrieval.
• Incorporated contrastive learning and hashing into model design for group-wise causal reasoning.
• Employed hash binary opposite values for contrastive dual sampling and data augmentation.
• Developed a novel strategy for constructing an interpretable UDCMH model via interventional training.

Main Results:

• The proposed TCICH model demonstrates superior effectiveness compared to most existing UDCMH methods.
• Experiments on three baseline datasets validate the model's performance.
• The framework successfully integrates contrastive learning and hashing for enhanced feature learning.

Conclusions:

• TCICH offers a novel and effective solution for unsupervised cross-modal retrieval tasks.
• The integration of contrastive learning and hashing leads to deeper semantic understanding and improved retrieval accuracy.
• The model's interpretability and comprehensive performance make it well-suited for cross-modal retrieval applications.