MaskTrack: Auto-Labeling and Stable Tracking for Video Object Segmentation.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Video object segmentation (VOS) has advanced with new datasets and architectures.
• Manual video mask annotation is labor-intensive and costly, limiting context understanding in current benchmarks.
• Existing VOS methods struggle with stable long-term tracking and instance discrimination in complex scenes.

Purpose of the Study:

• To develop a cost-effective, zero-shot auto-labeling strategy for dense video instance annotation.
• To introduce a novel framework, MaskTrack, for robust long-term VOS and improved instance discrimination.
• To evaluate the proposed method's performance on various VOS and related tasks without pre-training on image datasets.

Main Methods:

• Implemented a zero-shot auto-labeling strategy leveraging the Segment Anything Model (SAM) for dense video annotation.
• Developed the MaskTrack framework designed for stable long-term VOS and distinguishing similar objects in complex videos.
• Conducted extensive experiments on established VOS benchmarks (YouTube-VOS, LVOS) and related tasks (VOT, RVOS).

Main Results:

• Achieved excellent performance on short-term VOS (86.2% YouTube-VOS val) and long-term VOS (68.2% LVOS val) without image pre-training.
• Demonstrated significant advantages in distinguishing instances within complex videos with densely packed similar objects.
• Showcased strong generalization capabilities, performing well on visual object tracking (65.6% VOTS2023) and referring VOS (65.2% Ref YouTube VOS).

Conclusions:

• The proposed zero-shot auto-labeling strategy and MaskTrack framework effectively address challenges in video object segmentation.
• The method simplifies annotation and improves performance in long-term and complex video scenarios.
• The approach exhibits strong generalization across different video analysis tasks.