TMMSAM2: Tracker-Aided Multitemporal Memory SAM2 for Hyperspectral Object Tracking.

Area of Science:

• Computer Vision
• Remote Sensing
• Machine Learning

Background:

• Segment Anything Model 2 (SAM2) excels at prompt-based segmentation but faces challenges in hyperspectral object tracking.
• Existing methods struggle with dynamic environments, occlusions, and the high dimensionality of hyperspectral data.

Purpose of the Study:

• To propose TMMSAM2, a novel SAM2-based framework for robust hyperspectral object tracking.
• To enhance tracking performance without requiring additional model training.
• To address limitations of traditional tracking strategies in complex scenarios.

Main Methods:

• Developed a multitemporal memory bank (MMB) integrating short-, medium-, and long-term temporal scales for optimal spatiotemporal diversity.
• Implemented a three-dimensional constraint error mechanism (TCEM) for real-time quality assessment based on velocity, size, and motion.
• Integrated a ViPT-based hyperspectral tracker for automatic correction of tracking anomalies.

Main Results:

• TMMSAM2 demonstrated superior performance in hyperspectral object tracking across three public datasets.
• The framework effectively handles complex scenarios with rapid target movement and frequent occlusions.
• The integrated quality assurance system ensures reliable tracking results.

Conclusions:

• TMMSAM2 offers a robust and efficient solution for hyperspectral object tracking, leveraging SAM2's capabilities.
• The novel MMB and TCEM components significantly improve tracking accuracy and reliability.
• This framework advances the state-of-the-art in hyperspectral imaging applications.