Conditional Feature Learning Based Transformer for Text-Based Person Search.

Area of Science:

• Computer Vision
• Natural Language Processing
• Machine Learning

Background:

• Text-based person search requires matching descriptive text to individuals in image galleries.
• Existing Transformer methods often use brute-force feature concatenation, leading to suboptimal alignment and feature distribution.
• Inferring latent correspondences between image sub-regions and text phrases at various scales is crucial but challenging.

Purpose of the Study:

• To develop a novel Transformer-based approach for enhanced text-based person search.
• To explicitly align image sub-region features and textual phrase features in a unified latent space.
• To introduce a multi-modal re-ranking method for further performance improvement.

Main Methods:

• Conditional Feature Learning based Transformer (CFLT) maps sub-regions and phrases into a unified latent space.
• Conditional embeddings dynamically adjust features from one modality based on the other, enabling explicit alignment.
• A Re-ranking scheme by Visual Conditional Feature (RVCF) is proposed for multi-modal re-ranking.

Main Results:

• CFLT achieves a superior feature distribution compared to previous methods.
• The proposed RVCF method significantly enhances performance by leveraging visual conditional features.
• CFLT outperforms state-of-the-art methods by 7.03% in top-1 accuracy and 5.01% in top-5 accuracy.

Conclusions:

• CFLT provides an effective mechanism for explicit cross-modal alignment in text-based person search.
• The combination of CFLT and RVCF offers substantial improvements in search accuracy.
• This work advances the field of text-based person retrieval through improved feature learning and re-ranking.