UniSOT: A Unified Framework for Multi-Modality Single Object Tracking.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Single object tracking (SOT) typically uses specific reference (e.g., bounding box, natural language) and video modalities (e.g., RGB, RGB+Depth).
• Existing SOT trackers are often modality-specific, leading to separate model designs and limited applicability.
• A unified approach is needed to handle diverse tracking requirements across various modalities.

Purpose of the Study:

• To introduce UniSOT, a novel unified tracker capable of handling multiple reference and video modalities simultaneously with uniform parameters.
• To address the limitations of existing modality-specific trackers in practical applications.

Main Methods:

• Developed UniSOT, a unified tracker designed for combinations of three reference modalities and four video modalities.
• Employed uniform parameters across all modality combinations.
• Conducted extensive experiments on 18 diverse benchmarks including visual tracking, vision-language tracking, and RGB+X tracking.

Main Results:

• UniSOT demonstrated superior performance compared to modality-specific trackers across various benchmarks.
• Achieved over 3.0% improvement in AUC on the TNL2K benchmark across all three reference modalities.
• Outperformed Un-Track by over 2.0% on the main metric across all three RGB+X video modalities.

Conclusions:

• UniSOT offers a versatile and high-performing solution for single object tracking across diverse reference and video modalities.
• The unified approach simplifies model design and enhances practical applicability in complex scenarios.
• UniSOT sets a new standard for multi-modal tracking performance.