Progressively Hybrid Transformer for Multi-Modal Vehicle Re-Identification.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Multi-modal vehicle re-identification (e.g., visible, near-infrared, thermal-infrared) is crucial for surveillance and security, especially in low-illumination conditions.
• Effective fusion of complementary information across different imaging modalities is a significant challenge due to varying imaging characteristics.

Purpose of the Study:

• To propose a novel method, the progressively hybrid transformer (PHT), for effective multi-modal complementary information fusion in vehicle re-identification.
• To enhance the accuracy and robustness of vehicle re-identification systems operating under diverse lighting conditions.

Main Methods:

• The proposed progressively hybrid transformer (PHT) incorporates two key components: random hybrid augmentation (RHA) and a feature hybrid mechanism (FHM).
• RHA utilizes an image random cropper and a local region hybrider to mitigate modal differences by fusing local structural characteristics from all modalities.
• FHM employs a modal-specific controller and modal information embedding for effective feature-level fusion of multi-modal information.

Main Results:

• The PHT method achieved superior performance compared to state-of-the-art methods.
• Demonstrated a 2.7% higher mean Average Precision (mAP) on the RGBNT100 dataset.
• Achieved a 6.6% higher mAP on the RGBN300 dataset, validating its effectiveness in learning multi-modal complementary information.

Conclusions:

• The proposed PHT method effectively learns and fuses complementary information from multiple imaging modalities for vehicle re-identification.
• The RHA and FHM components significantly contribute to mitigating modal differences and improving feature-level fusion.
• The method shows strong potential for practical applications requiring robust vehicle re-identification in challenging, low-illumination environments.