Dual-Level Modality De-Biasing for RGB-T Tracking.

Area of Science:

• Computer Vision
• Machine Learning
• Robotics

Background:

• Robust tracking requires integrating Red Green Blue (RGB) and Thermal Infrared (TIR) data.
• Existing RGB-T trackers exhibit biases due to reliance on RGB pre-trained models and modality response inconsistencies.
• These biases manifest as feature-level and decision-level disparities in TIR modality processing.

Purpose of the Study:

• To propose a novel dual-level modality de-biasing framework for RGB-T tracking.
• To eliminate feature-level and decision-level biases in the TIR modality.
• To enhance the robustness and accuracy of RGB-T tracking systems.

Main Methods:

• A joint infrared-fusion adapter, including an infrared-aware adapter and a cross-fusion adapter, was developed to mitigate feature-level biases and fuse complementary information.
• A response-decoupled distillation strategy was introduced to explicitly address decision-level biases.
• The framework aims for adaptive bias mitigation and consistent decision-making between RGB and TIR modalities.

Main Results:

• The proposed framework effectively mitigates both feature-level and decision-level biases in RGB-T tracking.
• Experimental validation on popular benchmarks demonstrates significant improvements in tracking performance.
• The method achieves more consistent and accurate decision-making by balancing RGB and TIR modality contributions.

Conclusions:

• The dual-level modality de-biasing framework offers a promising solution for overcoming inherent biases in RGB-T tracking.
• The proposed adapters and distillation strategy effectively leverage complementary information while ensuring modality balance.
• This work advances the state-of-the-art in robust multi-modal visual tracking.