Semantic-Aware Multimodal Collaborative Learning for Unsupervised Visible-Infrared Person Re-Identification

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Unsupervised visible-infrared person reidentification (VI-ReID) faces challenges due to the modality gap between visible and infrared images.
• Existing methods often use noisy one-hot pseudo-labels, limiting semantic understanding and robustness.

Purpose of the Study:

• To propose a novel Semantic-aware Multimodal Collaborative Learning (SAMCL) framework for unsupervised VI-ReID.
• To address the limitations of existing methods by enhancing cross-modal and intra-modality feature learning.

Main Methods:

• Developed a Modality-aware Semantic Fusion (MSF) module to integrate complementary semantic details across modalities, creating enriched cross-modal supervision.
• Introduced a Dynamic Contrastive Learning (DCL) module for refined intra-modality feature learning by aligning samples with dynamic centroids.
• Combined MSF and DCL modules within the SAMCL framework for robust unsupervised VI-ReID.

Main Results:

• Achieved state-of-the-art (SOTA) performance on multiple benchmark datasets.
• On SYSU-MM01 (All Search), achieved 68.68% Rank-1 accuracy, surpassing SOTA by 3.48%.
• On RegDB (Visible-to-Infrared), achieved 94.47% Rank-1 accuracy, outperforming SOTA by 3.57%.
• On LLCM (Visible-to-Infrared), achieved 50.6% Rank-1 accuracy, outperforming SOTA by 3.7%.

Conclusions:

• The proposed SAMCL framework effectively overcomes the modality gap in unsupervised VI-ReID.
• SAMCL demonstrates superior performance and robustness compared to existing methods.
• The framework minimizes reliance on noisy pseudo-labels through effective multimodal collaboration.