Pseudo Sentences Evaluation and Quality-Aware Robust Learning for Unsupervised Text-Based Person Search

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Unsupervised Text-Based Person Search (TBPS) relies on pseudo-sentences generated by Multi-modal Large Language Models (MLLMs).
• The quality of these pseudo-sentences can be poor, leading to semantic misalignment and hindering model performance.
• Existing methods lack robustness in handling imperfect pseudo-sentence data.

Purpose of the Study:

• To propose a unified framework, PSE-QRL, for enhancing robustness to pseudo-sentences in unsupervised TBPS.
• To improve the accuracy and reliability of representation learning in TBPS by addressing pseudo-sentence quality issues.
• To achieve state-of-the-art retrieval performance in unsupervised TBPS.

Main Methods:

• Developed PSE-QRL, a framework dynamically coupling an evolving TBPS model with MLLMs for pseudo-sentence reliability assessment.
• Implemented Multi-granularity Sentence Augmentation to increase the diversity of image-sentence pairs.
• Utilized Hybrid Quality Evaluation combining MLLM reasoning and TBPS distinguishing capabilities for sentence quality assessment.
• Employed Quality-aware Robust Learning to select and re-weight samples based on quality scores.

Main Results:

• Demonstrated the effectiveness of PSE-QRL in improving learning robustness for unsupervised TBPS.
• Achieved state-of-the-art (SOTA) retrieval performance on CUHK-PEDES, ICFG-PEDES, and RSTPReid benchmarks.
• Showcased significant improvements in handling semantic misalignment caused by low-quality pseudo-sentences.

Conclusions:

• The proposed PSE-QRL framework effectively enhances robustness to pseudo-sentences in unsupervised TBPS.
• Hybrid quality evaluation and quality-aware learning are crucial for leveraging pseudo-sentences effectively.
• PSE-QRL represents a significant advancement in unsupervised TBPS, offering SOTA performance and improved reliability.