Enhancing cross-modal retrieval via label graph optimization and hybrid loss functions.

Area of Science:

• Artificial Intelligence
• Multimedia Analysis
• Computer Vision

Background:

• Cross-modal retrieval, especially image-text matching, is vital for AI applications.
• Existing methods often fail to leverage semantic label relationships, limiting performance.
• Enhanced discriminability is needed for more effective multimedia analysis.

Purpose of the Study:

• To improve image-text matching by modeling label correlations.
• To enhance the discriminability and alignment of cross-modal retrieval systems.
• To introduce a novel approach addressing limitations in current retrieval methods.

Main Methods:

• Developed a Two-Layer Graph Convolutional Network (L2-GCN) to capture label correlations.
• Proposed a hybrid Circle-Soft loss function for improved alignment and discriminability.
• Conducted experiments on benchmark datasets: NUS-WIDE, MIRFlickr, and MS-COCO.

Main Results:

• The L2-GCN with Circle-Soft consistently outperformed existing baselines.
• Achieved accuracy improvements of 0.5% on NUS-WIDE and MIRFlickr.
• Demonstrated a 1.0% accuracy increase on the MS-COCO dataset.

Conclusions:

• The proposed L2-GCN and Circle-Soft approach effectively models label correlations for superior image-text matching.
• This method offers enhanced alignment and discriminability in cross-modal retrieval.
• The approach shows significant improvements on multiple large-scale datasets.