SAMURAI: Motion-Aware Memory for Training-Free Visual Object Tracking With SAM 2.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• The Segment Anything Model 2 (SAM 2) excels at object segmentation but struggles with visual object tracking, especially in crowded or occluded scenarios.
• SAM 2's fixed memory mechanism accumulates errors during occlusions, leading to inaccurate tracking and identity drift.
• Existing methods often require extensive retraining or fine-tuning to adapt segmentation models for tracking tasks.

Purpose of the Study:

• To introduce SAMURAI, an improved adaptation of SAM 2 designed for robust visual object tracking.
• To address the limitations of SAM 2 in handling complex tracking scenarios like occlusions and crowded scenes.
• To develop a training-free tracking method that leverages temporal motion cues and an optimized memory selection strategy.

Main Methods:

• SAMURAI integrates temporal motion cues with a novel motion-aware memory selection strategy.
• The model predicts object motion and refines mask selection dynamically.
• No retraining or fine-tuning of the base SAM 2 model is required.

Main Results:

• SAMURAI demonstrates strong training-free performance across multiple VOT benchmark datasets.
• Achieved state-of-the-art results on LaSOT_ext, GOT-10k, and TrackingNet benchmarks.
• Delivered competitive performance on LaSOT, VOT2020-ST, VOT2022-ST, and SA-V benchmarks.

Conclusions:

• SAMURAI offers a robust and precise solution for visual object tracking, overcoming SAM 2's limitations.
• The motion-aware memory selection strategy enhances tracking accuracy in complex dynamic environments.
• SAMURAI shows significant potential for real-world applications requiring reliable object tracking.