Soft Supervision Guided Spatial-Temporal Refinement Network For Video-based Visible-Infrared Person...

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Person re-identification (Re-ID) enables tracking individuals across cameras using visible and infrared modes.
• Video-based Re-ID offers richer appearance details than still images but faces challenges in capturing fine-grained information.
• Existing methods often lose details and struggle with intra-class variations, limiting model generalization.

Purpose of the Study:

• To develop a novel network for video-based cross-modal person Re-ID.
• To address the loss of fine-grained details in temporal representations.
• To improve model generalization by overcoming limitations of traditional metric losses.

Main Methods:

• Introduction of the Soft Supervision guided Spatial-Temporal Refinement (S3TR) network.
• Frame refinement guided by coarse temporal features for discriminative feature extraction.
• Global-local mutual learning to bridge the modality gap and a novel soft-clustering center loss for improved generalization.

Main Results:

• The proposed S3TR network effectively refines features and captures fine-grained details.
• The global-local mutual learning module successfully reduces the modality gap.
• The soft-clustering center loss enhances model generalization by considering group-wise similarities.

Conclusions:

• S3TR achieves superior performance in video-based cross-modal person Re-ID.
• The HITSZ-PVCM dataset is the largest to date for this task.
• The proposed methods offer a significant advancement in person re-identification accuracy and generalization.