Divert More Attention to Vision-Language Object Tracking.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Multimodal vision-language (VL) learning advances generic intelligence via foundation models.
• Object tracking, a core vision task, has lagged in benefiting from VL advancements due to data and methodology limitations.

Purpose of the Study:

• To address the scarcity of large-scale vision-language annotated videos for tracking.
• To develop effective vision-language representations and interaction learning for improved tracking.
• To construct a comprehensive VL tracking database to facilitate model learning.

Main Methods:

• A general attribute annotation strategy was employed to create a large-scale VL tracking database (>23,000 videos).
• A novel framework featuring asymmetric architecture search and a modality mixer (ModaMixer) was introduced for unified-adaptive VL representation.
• A contrastive loss was utilized to align different modalities, enhancing VL representation.

Main Results:

• The proposed framework was integrated with CNN-based (SiamCAR), Transformer-based (OSTrack), and hybrid (TransT) tracking methods.
• Significant performance improvements were observed across all integrated baseline methods on six tracking benchmarks.
• Theoretical analysis validated the rationality and effectiveness of the proposed approach.

Conclusions:

• The developed framework demonstrates the significant potential of vision-language representation for advancing object tracking.
• The creation of a large-scale VL tracking dataset facilitates further research in this domain.
• This work encourages greater community focus on VL tracking and multimodal integration for future tracking systems.