Memorize, Associate and Match: Embedding Enhancement via Fine-Grained Alignment for Image-Text Retrieval

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Current image-text retrieval methods face limitations in capturing fine-grained semantic correlations.
• Embedding learning paradigms struggle with detailed correspondence, while pair-wise methods are computationally expensive.

Purpose of the Study:

• To propose a novel method, MEMBER, for efficient and accurate image-text retrieval.
• To address the limitations of existing methods by enabling fine-grained alignment within the embedding learning framework.

Main Methods:

• MEMBER utilizes global memory banks to enrich image and text features.
• It achieves mutual embedding enhancement across modalities by incorporating relevant cross-modal features.
• This approach integrates fine-grained alignment and fusion into the embedding learning paradigm.

Main Results:

• MEMBER demonstrates superior performance compared to state-of-the-art methods.
• The method achieves remarkable improvements on two large-scale benchmark datasets.
• Experimental results validate the effectiveness of memory-based embedding enhancement.

Conclusions:

• MEMBER offers an efficient and effective solution for image-text retrieval.
• The proposed approach successfully combines fine-grained alignment with retrieval efficiency.
• This work advances the field of multimodal retrieval through innovative memory bank integration.