RGBT Tracking via Multi-Adapter Network with Hierarchical Divergence Loss

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• RGB-Thermal (RGBT) tracking combines visible light and thermal infrared data for robust, all-weather object tracking.
• Existing RGBT tracking methods often underutilize the complementary strengths of RGB and thermal modalities.
• There is a need for advanced methods to effectively learn from both shared and modality-specific features in RGBT tracking.

Purpose of the Study:

• To propose a novel multi-adapter network for joint learning of modality-shared, modality-specific, and instance-aware target representations in RGBT tracking.
• To enhance the robustness and accuracy of RGBT trackers by better exploiting the complementary advantages of RGB and thermal data.
• To develop an end-to-end deep learning framework that efficiently extracts multilevel representations.

Main Methods:

• A novel multi-adapter network architecture is proposed, comprising a generality adapter (modified VGG-M) for shared features, a modality adapter for efficient specific features, and an instance adapter for target-specific properties.
• The network is trained end-to-end, integrating multiple kernel maximum mean discrepancy (MKM-MD) loss to minimize distribution divergence between modal features.
• The modality adapter design allows for learning multilevel modality-specific representations with shared parameters, reducing computational complexity.

Main Results:

• The proposed RGBT tracker demonstrates outstanding performance compared to state-of-the-art methods on two benchmark datasets.
• The multi-adapter approach effectively captures both shared and modality-specific target characteristics.
• The integration of MKM-MD loss contributes to more robust representation learning by enhancing feature distribution alignment.

Conclusions:

• The novel multi-adapter network provides a superior approach for RGBT tracking by effectively learning comprehensive target representations.
• The proposed method achieves significant improvements in tracking accuracy and robustness, particularly in challenging all-day, all-weather conditions.
• This work highlights the potential of jointly learning shared, specific, and instance-aware features for advancing RGBT tracking technology.