Harnessing Knowledge From Pretrained VLMs for Unsupervised Person Search

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Person search combines pedestrian detection and re-identification, crucial for surveillance and robotics.
• Labeling data for supervised training is costly and time-consuming.
• Unsupervised person search is desirable but challenged by noisy pseudo-labels from low-quality detections.

Purpose of the Study:

• To develop a novel unsupervised person search method (FMUPS) that leverages semantic information for reliable pseudo-label generation.
• To address the challenges of inaccurate bounding boxes and misclassifications in unsupervised learning for person search.
• To improve the performance of person re-identification by mitigating noise in pseudo-labels.

Main Methods:

• Utilizing vision-language models (VLMs) for semantic representations to guide pseudo-label extraction and reduce background noise.
• Introducing an anti-bounding-box-noise re-identification loss to correct localization and classification errors.
• Developing a CLIP ID labeler that uses text-image alignment for pseudo-ID generation and refinement.

Main Results:

• FMUPS effectively generates reliable pseudo-labels by leveraging semantic information from VLMs.
• The anti-bounding-box-noise re-ID loss successfully alleviates localization and classification noise, enhancing re-ID feature learning.
• Experimental results on CUHK-SYSU and PRW benchmarks demonstrate superior performance compared to previous unsupervised and weakly supervised methods.

Conclusions:

• The proposed FMUPS method significantly improves unsupervised person search by effectively utilizing semantic information.
• Leveraging VLMs and specialized loss functions offers a promising direction for robust person search in real-world scenarios.
• The method demonstrates the potential for accurate person search without the need for extensive manual labeling.