Associate Everything Detected: Facilitating Tracking-by-Detection to the Unknown.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Multi-object tracking (MOT) is crucial in computer vision.
• Existing methods are either closed-vocabulary (CV-MOT) or open-vocabulary (OV-MOT), with limitations in the other's domain.
• A unified approach is needed to handle both predefined and unknown object categories.

Purpose of the Study:

• To present a unified framework, Associate Everything Detected (AED), for simultaneous CV-MOT and OV-MOT.
• To enable tracking of unknown object categories without prior knowledge.
• To improve tracking performance across diverse datasets and scenarios.

Main Methods:

• Developed a unified framework, Associate Everything Detected (AED).
• Modeled object association as a similarity decoding problem using a sim-decoder.
• Employed association-centric learning leveraging spatial, temporal, and cross-clip similarities.
• Integrated with off-the-shelf object detectors.

Main Results:

• AED achieves superior performance on TAO, SportsMOT, and DanceTrack datasets.
• The framework successfully handles complex trajectories in OV-MOT tasks.
• Maintained excellent performance in CV-MOT tasks without relying on prior knowledge like motion cues.

Conclusions:

• AED provides a unified and robust solution for both CV-MOT and OV-MOT.
• The proposed sim-decoder and association-centric learning effectively extract features for continuous and generalized tracking.
• AED demonstrates strong generalization capabilities for tracking unknown object categories.