Text-Visible/Infrared Person Retrieval: Attribute-Guided Feature Decoupling and Collaborative Alignment and a Unified...

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Current text-to-image person retrieval methods are limited to visible imagery, failing in low-light conditions.
• Infrared imaging is crucial for many visual systems, necessitating text matching across visible and infrared modalities.
• Heterogeneous visual characteristics of visible and infrared images pose challenges for unified cross-modal retrieval frameworks.

Purpose of the Study:

• To introduce a new task: Text-Visible/Infrared person retrieval.
• To propose a novel network, ACANet, for unified text-to-visible/infrared image matching.
• To establish a new benchmark dataset for advancing this research area.

Main Methods:

• Developed the Attribute-guided feature decoupling and Collaborative Alignment Network (ACANet) for unified cross-modal alignment.
• Decoupled visible image color and texture features, integrating them with infrared features for enhanced alignment.
• Extended masked language modeling to a cross-modal paradigm for fine-grained alignment across multiple image modalities.

Main Results:

• The proposed ACANet demonstrated superior performance compared to state-of-the-art methods on the new MM01LLCM-Text dataset.
• Successfully addressed the challenge of matching text descriptions to heterogeneous visible and infrared images.
• Validated the effectiveness of feature decoupling and cross-modal alignment strategies.

Conclusions:

• ACANet provides an effective unified framework for Text-Visible/Infrared person retrieval.
• The developed MM01LLCM-Text dataset facilitates further research in this domain.
• This work advances the capability of person retrieval in challenging low-light and multi-modal scenarios.