M⁵L: Multi-Modal Multi-Margin Metric Learning for RGBT Tracking.

Area of Science:

• Computer Vision
• Machine Learning
• Robotics

Background:

• Robust classification of hard samples is a significant challenge in Red-Green-Blue-Depth (RGBT) tracking.
• Existing methods often overlook the crucial inter-relations among multilevel hard samples, impacting tracking robustness.

Purpose of the Study:

• To propose a novel Multi-Modal Multi-Margin Metric Learning (M⁵L) framework for RGBT tracking.
• To enhance feature embedding robustness by considering the relationships between normal and hard positive/negative samples.

Main Methods:

• Developed a M⁵L framework that categorizes samples into normal positive, normal negative, hard positive, and hard negative.
• Designed a multi-modal multi-margin structural loss to preserve multilevel hard sample relations during training.
• Integrated an attention-based fusion module for quality-aware fusion of multi-modal data.

Main Results:

• The M⁵L framework demonstrated improved robustness in feature embeddings by leveraging sample hierarchies.
• The attention-based fusion module enabled effective integration of RGBT data.
• Experiments on large-scale datasets confirmed superior tracking performance compared to state-of-the-art methods.

Conclusions:

• The proposed M⁵L framework effectively addresses the challenge of hard sample classification in RGBT tracking.
• Considering the relations of multilevel hard samples is key to improving tracking robustness.
• The M⁵L framework offers a promising approach for advancing RGBT tracking technology.