UCPM: Uncertainty-Guided Cross-Modal Retrieval With Partially Mismatched Pairs.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Manual annotation for cross-modal retrieval (CMR) is labor-intensive.
• Internet data collection for CMR introduces Partially Mismatched Pairs (PMPs), degrading performance.
• Existing methods for filtering PMPs are sensitive and yield sub-optimal results.

Purpose of the Study:

• To propose an efficient approach, Uncertainty-guided Cross-modal retrieval with Partially Mismatched pairs (UCPM), to mitigate the impact of PMPs.
• To enhance the accuracy and robustness of cross-modal retrieval systems dealing with imperfect data.

Main Methods:

• Developed an Uncertainty Guided Division (UGD) strategy to categorize training data into clean, noisy, and hard subsets based on uncertainty.
• Implemented an Uncertainty Self-Correction (USC) mechanism to stabilize and refine uncertainty estimation during training.
• Introduced a Trusted Margin Loss (TML) to improve feature discriminability by adjusting margins for hard pairs.

Main Results:

• UCPM significantly reduces the adverse effects of mismatched data pairs in cross-modal retrieval.
• Experiments on benchmark datasets demonstrate UCPM's effectiveness and reliability compared to state-of-the-art approaches.
• The proposed method shows improved robustness against both synthetic and real-world PMPs.

Conclusions:

• UCPM offers an effective solution for handling Partially Mismatched Pairs in cross-modal retrieval.
• The uncertainty-guided approach enhances learning from informative pairs while minimizing noise from mismatched ones.
• UCPM advances the state-of-the-art in robust cross-modal retrieval.