CLIP-Based Adaptive Graph Attention Network for Large-Scale Unsupervised Multi-Modal Hashing Retrieval

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Unsupervised multi-modal hashing retrieval is crucial for efficient data management and retrieval.
• Existing methods struggle with capturing cross-modal information and preserving semantic structures during binarization.

Purpose of the Study:

• To address limitations in current unsupervised multi-modal hashing retrieval methods.
• To develop a robust method for accurate similarity measurement and discriminative hash code generation.

Main Methods:

• Utilized CLIP for fine-grained semantic feature extraction and similarity fusion.
• Developed an adaptive graph attention network with graph convolutional networks for enhanced hash code learning.
• Employed an iterative approximate optimization strategy to minimize information loss during binarization.

Main Results:

• The proposed CLIP-based Adaptive Graph Attention Network (CAGAN) significantly improves retrieval accuracy.
• CAGAN effectively captures complementary and co-occurrence information across modalities.
• Demonstrated superior performance over existing methods on benchmark datasets.

Conclusions:

• CAGAN offers a powerful solution for large-scale unsupervised multi-modal hashing retrieval.
• The method effectively balances multi-modal learning and preserves data semantic structure.
• CAGAN shows significant potential for real-world applications requiring efficient multi-modal data retrieval.