Object Tracking in RGB-T Videos Using Modal-Aware Attention Network and Competitive Learning

Area of Science:

• Computer Vision
• Artificial Intelligence
• Sensor Fusion

Background:

• Object tracking in RGB-thermal (RGB-T) videos is vital for all-weather, all-day applications.
• Effective fusion of RGB and thermal data remains a significant challenge for robust tracking.

Purpose of the Study:

• To propose a novel RGB-T object tracking algorithm, MaCNet, that enhances dual-modality information fusion.
• To improve the robustness and performance of RGB-T object trackers.

Main Methods:

• A two-stream network for feature extraction from RGB and thermal images.
• A modal-aware attention network to guide feature fusion and enhance inter-modality information interaction.
• A modality-egoistic loss function with competitive learning for network fine-tuning.

Main Results:

• The proposed MaCNet tracker demonstrates superior performance over state-of-the-art RGB-T and RGB trackers on public datasets.
• The modal-aware attention mechanism effectively integrates and leverages information from both modalities.
• Competitive learning optimizes the fusion strategy for enhanced tracking accuracy.

Conclusions:

• MaCNet offers a robust and effective solution for RGB-T object tracking by intelligently fusing dual-modality information.
• The proposed approach advances the field of multi-modal object tracking, particularly in challenging environmental conditions.