Jointly Modeling Motion and Appearance Cues for Robust RGB-T Tracking

Area of Science:

• Computer Vision
• Machine Learning
• Robotics

Background:

• Robust object tracking is crucial for various applications.
• Existing RGB-T trackers often struggle with appearance variations or motion complexities.
• Integrating complementary cues from different modalities is an active research area.

Purpose of the Study:

• To develop a novel RGB-T tracking framework that robustly handles appearance and motion variations.
• To improve tracking performance by effectively fusing RGB and thermal data.
• To enhance tracker adaptability through flexible switching between appearance and motion models.

Main Methods:

• A late fusion strategy is proposed to dynamically infer fusion weights for RGB and thermal modalities.
• Offline-trained global and local multimodal fusion networks determine these weights for robust appearance modeling.
• Target and camera motion cues are integrated to ensure tracker robustness when appearance information is unreliable.
• A tracker switcher mechanism is introduced for flexible adaptation between appearance and motion-based tracking.

Main Results:

• The proposed framework demonstrates superior performance on three recent RGB-T tracking datasets.
• The late fusion method effectively combines RGB and thermal appearance information.
• The integrated motion cues significantly improve tracking robustness in challenging scenarios.
• The tracker switcher enhances adaptability and overall tracking accuracy.

Conclusions:

• The novel RGB-T tracking framework offers significant improvements over state-of-the-art methods.
• Jointly modeling appearance and motion cues leads to more robust and accurate tracking.
• The proposed fusion and switching strategies provide a flexible and effective approach for RGB-T tracking.