QueryTrack: Joint-Modality Query Fusion Network for RGBT Tracking.

Area of Science:

• Computer Vision
• Machine Learning
• Robotics

Background:

• Existing RGB-Thermal (RGB-T) trackers often process feature extraction and fusion separately, limiting performance.
• This separation neglects the synergistic potential between modalities and adaptability to varying quality.

Purpose of the Study:

• To develop an integrated approach for RGB-T tracking that couples feature extraction and fusion.
• To enhance the exploitation of complementary RGB-T information and improve robustness to modal quality fluctuations.

Main Methods:

• A joint-modality query fusion (JQF) network is proposed, coupling intra-modal feature extraction and inter-modal fusion.
• Joint-modality queries, initialized from current frame multimodal features, drive adaptive fusion.
• Regional cross-attention is employed for efficient cross-modal interactions, enabling real-time performance.

Main Results:

• The JQF network effectively unifies intra-modal enhancement and inter-modal interactions.
• The tracker demonstrates state-of-the-art performance on multiple RGBT tracking benchmarks (LasHeR, VTUAV, RGBT234, GTOT).
• The proposed method achieves real-time tracking speeds.

Conclusions:

• The JQF network offers a more effective and adaptive solution for RGB-T tracking.
• Coupling feature extraction and fusion significantly improves performance and robustness.
• The tracker achieves a new state-of-the-art, balancing accuracy and speed.