Text-Enhanced Graph Attention Hashing for Cross-Modal Retrieval

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Deep hashing enables efficient storage and retrieval, crucial for growing multimodal data.
• Existing methods struggle with noisy labels and multiscale features, increasing information entropy and reducing retrieval accuracy.
• Variations in text data increase information entropy during feature extraction, hindering cross-modal retrieval performance.

Purpose of the Study:

• To develop a novel framework, Text-Enhanced Graph Attention Hashing (TEGAH), for effective cross-modal retrieval.
• To minimize information entropy in hash code and feature generation.
• To enhance the retrieval efficiency of large-scale media data.

Main Methods:

• Implemented a deep text feature extraction network and a multiscale label region fusion network to reduce information entropy.
• Utilized a Graph-Attention-based modal feature fusion network for integrating multimodal information.
• Designed TEGAH to improve network affinity across different modalities and preserve semantic information.

Main Results:

• The TEGAH framework demonstrated significant improvements in cross-modal retrieval performance.
• Experiments were conducted on three multilabel datasets, validating the framework's effectiveness.
• TEGAH outperformed existing state-of-the-art cross-modal hashing methods.

Conclusions:

• TEGAH effectively addresses challenges of noisy labels, multiscale features, and information entropy in cross-modal retrieval.
• The proposed framework enhances feature extraction and fusion for better retrieval accuracy.
• TEGAH offers a promising solution for efficient and accurate large-scale media data retrieval.