Improvement of SAM2 Algorithm Based on Kalman Filtering for Long-Term Video Object Segmentation.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Segment Anything Model 2 (SAM2) excels at pixel-level segmentation but struggles with long-term video tracking due to its static inference and fixed temporal window.
• Error propagation and tracking instability occur in SAM2 with fast-moving objects, occlusions, or crowded scenes.

Purpose of the Study:

• To introduce SAM2Plus, a zero-shot enhancement framework designed to overcome SAM2's limitations in long-term video object segmentation.
• To improve tracking accuracy, stability, and robustness in dynamic and challenging visual environments.

Main Methods:

• Integration of Kalman filter prediction for motion modeling and trajectory refinement.
• Implementation of dynamic quality thresholds and multi-criteria evaluation for adaptive frame selection.
• Development of an optimized memory system with adaptive pruning and context retention.

Main Results:

• SAM2Plus demonstrated superiority over SAM2, achieving an average improvement of 1.0 in J&F metrics.
• Significant gains exceeding 2.3 points were observed on SA-V and LVOS datasets for long-term tracking.
• Real-time performance and strong generalization were achieved without fine-tuning, effectively handling occlusions and viewpoint changes.

Conclusions:

• SAM2Plus effectively unifies motion-aware prediction with spatial segmentation, bridging static and dynamic reasoning gaps.
• The framework offers a scalable solution for real-world applications like autonomous driving and surveillance.
• SAM2Plus provides enhanced robustness and accuracy for video object segmentation tasks.