Explicit semantic guided bi-incomplete multi-modal hashing with label co-occurrence and label graph constraints

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Multi-modal hashing enhances large-scale multimedia retrieval by integrating diverse features into compact binary codes.
• Existing methods struggle with incomplete multi-modal features and labels, particularly at high missing rates.

Purpose of the Study:

• To propose LaDiff-BIMH, a unified framework for bi-incomplete multi-modal hashing that addresses missing features and labels.
• To improve the accuracy and efficiency of multimedia retrieval in scenarios with incomplete data.

Main Methods:

• LaDiff-BIMH employs a three-stage approach: Label Graph Constrained Autoencoder for modal reconstruction, Conditional DDPM for incomplete modal completion, and Explicit Semantic guided Multi-modal Hash Learning.
• Leverages label co-occurrence and graph constraints for feature reconstruction and pseudo-label generation.
• Utilizes adaptive weighted fusion and a discriminative hash center for enhanced hash code generation.

Main Results:

• LaDiff-BIMH demonstrates superior performance compared to state-of-the-art methods in multi-modal hashing.
• The framework effectively handles bi-incompleteness in both features and labels.
• Improved semantic consistency and discriminability of generated hash codes.

Conclusions:

• LaDiff-BIMH offers a robust solution for multi-modal hashing with incomplete data.
• The proposed framework significantly advances the field of multimedia retrieval under challenging data conditions.
• Future work may explore further enhancements in handling complex missing data patterns.