Active Supervised Cross-Modal Retrieval.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Supervised Cross-Modal Retrieval (SCMR) relies heavily on large, annotated multi-modal datasets, limiting its practical application.
• Existing Active Learning (AL) methods for SCMR often ignore inter-modality relationships and lead to biased sample selection, degrading performance.

Purpose of the Study:

• To develop an Active Supervised Cross-Modal Retrieval (ASCMR) framework that effectively identifies informative multi-modal samples and ensures unbiased selections.
• To reduce the annotation cost for SCMR while maintaining high retrieval performance.

Main Methods:

• Proposed a probabilistic multi-modal informativeness estimation to capture intra-modality and inter-modality uncertainty within a unified representation.
• Introduced a density-aware budget allocation strategy with semantic density regularization for unbiased sample selection.

Main Results:

• Evaluated on MS-COCO, NUS-WIDE, and MIRFlickr datasets, demonstrating significant annotation cost reduction.
• Achieved over 95% of fully supervised performance using only 6%, 3%, and 4% of active selected samples, respectively.
• Outperformed existing active learning strategies in reducing annotation requirements.

Conclusions:

• The proposed ASCMR framework effectively addresses the limitations of existing AL methods for SCMR.
• ASCMR enables high retrieval performance with substantially reduced labeling efforts, making SCMR more practical.