Nystromformer based cross-modality transformer for visible-infrared person re-identification.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Traditional person re-identification (Re-ID) struggles in varying illumination and at night.
• Visible-infrared (VIS-IR) Re-ID leverages infrared for low-light conditions but faces cross-modality discrepancies.
• Effective feature fusion between visible and infrared data remains a significant challenge.

Purpose of the Study:

• To develop a robust framework for VIS-IR person re-identification.
• To address and overcome the cross-modality discrepancies between visible and infrared images.
• To improve the accuracy and reliability of person re-identification systems in diverse environmental conditions.

Main Methods:

• Proposed NiCTRAM: a Nyströmformer-based Cross-Modality Transformer.
• Utilized a shared CNN backbone for hierarchical feature extraction from RGB and IR images.
• Employed parallel Nyströmformer encoders for efficient long-range dependency capture.
• Introduced a cross-attention fusion block integrating second-order covariance statistics for feature alignment.

Main Results:

• NiCTRAM achieved state-of-the-art performance on benchmark VIS-IR person Re-ID datasets.
• Demonstrated significant improvements over existing methods, surpassing SOTA by up to 5.90% in Rank-1 accuracy and 5.83% in mAP.
• Showcased robustness in handling cross-modality challenges inherent in VIS-IR Re-ID.

Conclusions:

• NiCTRAM effectively bridges the modality gap in VIS-IR person re-identification.
• The proposed framework offers a robust and accurate solution for challenging surveillance scenarios.
• The method provides significant advancements for practical applications requiring reliable person tracking across different sensors.