Language Supervised Multi-Camera Multi-Object Tracking.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Multi-camera multi-object tracking (MC-MOT) typically requires complex per-detection identity annotations.
• Language descriptions offer a more intuitive and human-friendly annotation alternative.

Purpose of the Study:

• To explore language-supervised MC-MOT (LS-MCMOT).
• To propose a novel approach, LaVST, for LS-MCMOT using weakly-supervised learning.
• To introduce an ID-aware projection self-correction mechanism.

Main Methods:

• Developed LaVST for language-to-vision weakly-supervised learning.
• Generated pseudo-labels via tracklet-level cross-modality matching.
• Implemented an ID-aware projection self-correction mechanism for improved accuracy.

Main Results:

• Trained models demonstrated promising performance in LS-MCMOT.
• Achieved favorable results compared to state-of-the-art identity-supervised methods.
• Showcased significant gains (20.0% IDF1) in cross-dataset evaluation, proving the efficacy of language annotations.

Conclusions:

• Language annotations hold significant potential for advancing MC-MOT.
• The proposed LaVST approach offers a viable and effective alternative to traditional identity supervision.
• LS-MCMOT methods can potentially outperform identity-supervised methods, especially in cross-dataset scenarios.