Channel Exchanging for RGB-T Tracking.

Area of Science:

• Computer Vision
• Machine Learning
• Sensor Fusion

Background:

• Single-modality visual object tracking struggles in diverse, all-weather conditions.
• RGB and thermal infrared (TIR) data offer complementary information for robust tracking.
• Existing RGB-T fusion methods often underutilize single-modality information or inter-modality alignment.

Purpose of the Study:

• To develop an advanced RGB-T object tracking framework that effectively fuses complementary data.
• To improve information utilization within single modalities and between RGB and TIR data.
• To enhance the generalization and long-term tracking capabilities of RGB-T trackers.

Main Methods:

• Proposed a novel RGB-T object tracking framework, Channel Exchanging DiMP (CEDiMP), based on the DiMP tracker.
• Implemented a dynamic channel exchanging mechanism between RGB and TIR sub-networks for feature fusion with minimal parameter overhead.
• Enhanced model generalization and long-term tracking by training on the synthetic LaSOT-RGBT dataset.

Main Results:

• CEDiMP demonstrated significantly stronger deep feature representation through its channel exchanging fusion strategy.
• The proposed method achieved state-of-the-art performance on the GTOT and RGBT234 RGB-T object tracking benchmark datasets.
• CEDiMP exhibited outstanding performance in generalization testing, indicating robust real-world applicability.

Conclusions:

• The channel exchanging mechanism is an effective approach for fusing RGB and TIR data in object tracking.
• CEDiMP offers a robust and generalizable solution for all-weather visual object tracking challenges.
• The findings highlight the potential of cross-modal feature fusion for advancing computer vision tasks.