LVOS: A Benchmark for Large-Scale Long-Term Video Object Segmentation.

Area of Science:

• Computer Vision
• Machine Learning

Background:

• Video object segmentation (VOS) models excel on short-term benchmarks.
• Current benchmarks do not represent real-world VOS challenges like long-term object visibility and reappearance.
• A need exists for datasets reflecting practical VOS scenarios.

Purpose of the Study:

• Introduce a novel benchmark, LVOS, for evaluating VOS models in realistic, long-term scenarios.
• Assess the performance of existing VOS models on challenging, real-world video data.
• Identify key factors hindering VOS performance in practical applications.

Main Methods:

• Developed the Long-term Video Object Segmentation (LVOS) benchmark with 720 videos, 296,401 frames, and 407,945 annotations.
• Included diverse attributes reflecting real-world challenges: long-term reappearance, cross-temporal similarity, and occlusion.
• Evaluated 15 state-of-the-art VOS models on LVOS under three settings.

Main Results:

• LVOS benchmark revealed a significant performance drop for existing VOS models.
• Increased video length, long-term reappearance, cross-temporal confusion, and occlusion were identified as major challenges.
• Models struggled with precise tracking and segmentation in long-duration, complex videos.

Conclusions:

• The LVOS benchmark effectively highlights the limitations of current VOS models in real-world applications.
• Long video duration and complex interactions significantly degrade VOS performance.
• LVOS is crucial for advancing VOS development for realistic scenarios.