Hybrid DAER Based Cross-Modal Retrieval Exploiting Deep Representation Learning.

Area of Science:

• Computer Science
• Artificial Intelligence
• Information Science

Background:

• Cross-modal retrieval aims to retrieve information across different data types like text and images.
• A key challenge is the heterogeneous gap between modalities, often addressed by common subspace methods.
• Existing methods often overlook the importance of fine-grained regional information within modalities.

Purpose of the Study:

• To propose a novel text-image cross-modal retrieval approach named DAER (Dual Attention and Enhanced Relation network).
• To address the limitations of existing methods by focusing on fine-grained regional importance.
• To improve the accuracy and effectiveness of cross-modal information retrieval.

Main Methods:

• Constructing a dual attention network to extract fine-grained weight information from both text and images.
• Developing an enhanced relation network to expand inter-category data differences.
• Improving computational accuracy of similarity calculations.

Main Results:

• The proposed DAER approach demonstrated effectiveness in cross-modal retrieval tasks.
• Experiments on Wikipedia, Pascal Sentence, and XMediaNet datasets confirmed superiority over existing methods.
• The dual attention and enhanced relation networks significantly improved retrieval performance.

Conclusions:

• The DAER approach offers a significant advancement in text-image cross-modal retrieval.
• Highlighting fine-grained regional importance is crucial for effective cross-modal information utilization.
• The proposed method provides a robust solution for bridging the heterogeneous gap in multimodal data.