Generative Retrieval for Unsupervised Text-Based Person Search

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Text-based person search (TBPS) typically requires supervised learning with annotated image-text pairs.
• Unsupervised TBPS methods are needed to leverage large unlabeled image datasets.

Purpose of the Study:

• To develop an unsupervised text-based person search (TBPS) framework.
• To generate fine-grained and diverse textual descriptions for images.
• To improve retrieval accuracy in TBPS using unlabeled data.

Main Methods:

• Proposed GTR+ framework: a two-stage generation-then-retrieval approach.
• Tiered description generation: automated Q&A, inter-sample contrastive learning, and stylized expansion.
• Adaptive confidence-weighted retrieval learning using a Gaussian Mixture Model to handle noisy pseudo-texts.

Main Results:

• Successfully implemented unsupervised TBPS using unlabeled images.
• Generated fine-grained and stylistically diverse textual descriptions.
• Developed an adaptive learning framework to mitigate noisy pseudo-texts, improving retrieval performance.

Conclusions:

• The GTR+ framework enables effective unsupervised TBPS.
• The proposed tiered description generation and adaptive retrieval learning enhance performance.
• Introduced LargeFine-Person dataset as a benchmark for unsupervised TBPS pre-training.