Revisiting Text-Based Person Retrieval: Mitigating Annotation-Induced Mismatches with Multimodal Large Language...

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Text-based person retrieval (TBPR) relies on high-quality benchmarks for accurate model evaluation.
• Current TBPR benchmarks suffer from ambiguous annotations, where similar images have identical descriptions, causing evaluation errors.
• Individual image annotation without considering similar images hinders the creation of distinctive descriptions.

Purpose of the Study:

• To propose an annotation refinement framework to enhance the quality of TBPR benchmarks.
• To mitigate annotation-induced mismatches in TBPR model evaluation.
• To improve the discriminative power of textual descriptions for person images.

Main Methods:

• Automated identification of image sets prone to mismatches using TBPR models.
• Leveraging multimodal large language models (MLLMs) to process multiple images simultaneously.
• Generating distinctive descriptions for each image and replacing original annotations to improve quality.

Main Results:

• The proposed framework effectively improves annotation quality in TBPR benchmarks.
• Experiments on CUHK-PEDES, RSTPReid, and ICFG-PEDES validate the method's effectiveness.
• More discriminative captions generated by the framework benefit mainstream TBPR models.

Conclusions:

• The annotation refinement framework significantly enhances TBPR benchmark quality.
• Improved benchmarks lead to more accurate evaluation of TBPR models.
• The enhanced benchmark datasets will be publicly released to benefit the research community.