Long-Tailed Approaching Cross-Modal Hashing With Multi-Expert Collaborative Learning

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Cross-modal hashing facilitates efficient retrieval across diverse data types by encoding them into binary codes.
• Existing methods struggle with real-world long-tailed (imbalanced) datasets, leading to poor performance.
• Current long-tailed cross-modal hashing methods are biased towards majority classes.

Purpose of the Study:

• To develop a novel cross-modal hashing framework that effectively handles imbalanced, long-tailed datasets.
• To address the performance bias of existing methods on minority classes in cross-modal retrieval.

Main Methods:

• Proposed a Long-tailed Approaching Cross-modal Hashing (LACH) framework utilizing multi-expert collaborative learning.
• Employed a Graph Convolutional Network (GCN) backbone for knowledge transfer and a partial data replication strategy for balanced expert training.
• Introduced a proxy-based pointwise loss for equitable category treatment and an inter-modal approaching loss for hash code alignment.

Main Results:

• LACH demonstrated significant accuracy improvements over state-of-the-art methods.
• Achieved up to 4.2% accuracy gain on balanced datasets and 6.3% on long-tailed datasets.
• The proposed methods effectively handle class imbalance in cross-modal hashing tasks.

Conclusions:

• LACH offers a robust solution for cross-modal hashing on imbalanced datasets.
• The multi-expert approach and novel loss functions contribute to equitable and accurate retrieval.
• This framework advances the field of efficient cross-modal data retrieval for real-world applications.