Relation-Aggregated Cross-Graph Correlation Learning for Fine-Grained Image-Text Retrieval.

Area of Science:

• Computer Vision
• Natural Language Processing
• Artificial Intelligence

Background:

• Fine-grained image-text retrieval is crucial for bridging vision and language modalities.
• Existing methods often overlook intermodal relations, limiting semantic correspondence learning.
• Learning fine-grained semantic correspondence across modalities remains a significant challenge.

Purpose of the Study:

• To propose a novel Relation-Aggregated Cross-Graph (RACG) model for enhanced fine-grained image-text retrieval.
• To explicitly learn fine-grained semantic correspondence by aggregating intramodal and intermodal relations.
• To improve the accuracy and performance of image-text retrieval tasks.

Main Methods:

• Constructing semantic-embedded graphs to represent objects and their intramodal relations.
• Designing a cross-graph relation encoder to capture intermodal relations.
• Utilizing feature reconstruction and multihead similarity alignment for node-level semantic correspondence optimization.

Main Results:

• The proposed RACG model effectively learns fine-grained semantic correspondence.
• The model achieves competitive performance on benchmark datasets for image-text retrieval.
• Quantitative and qualitative experiments validate the framework's advantages.

Conclusions:

• The RACG model offers a significant advancement in fine-grained image-text retrieval.
• Integrating intramodal and intermodal relations enhances semantic understanding across modalities.
• The framework demonstrates state-of-the-art performance and broad applicability.