Hyperspectral Video Tracking With Spectral-Spatial Fusion and Memory Enhancement

Yuzeng Chen , Qiangqiang Yuan , Hong Xie

IEEE Transactions on Image Processing : a Publication of the IEEE Signal Processing Society +

|

May 19, 2025

View abstract on PubMed

Summary

SpectralTrack enhances hyperspectral video (HSV) tracking by fusing spectral-spatial information and memory. This novel framework addresses data scarcity and computational load, outperforming existing HSV tracking methods.

Area Of Science

Computer Vision
Remote Sensing
Signal Processing

Background

Hyperspectral video (HSV) offers rich spectral-spatial-temporal data for advanced object tracking.
Existing HSV tracking methods struggle with data scarcity, band gaps, spectral fragmentation, and high computational costs.

Purpose Of The Study

To introduce SpectralTrack, a novel framework for robust and efficient hyperspectral video tracking.
To address limitations of current HSV tracking methods, including data scarcity and computational overhead.

Main Methods

SpectralTrack employs spectral-spatial fusion and memory enhancement for improved tracking.
A visual prompting module mitigates band gaps and spectral fragmentation.
An extraction-matching-interaction module with adapters in a Transformer architecture enables efficient feature processing.
A memory perception module refines spectral and spatial cues using temporal prompts.

Main Results

SpectralTrack demonstrates superior performance across nine HSV tracking datasets.
The framework effectively alleviates data scarcity and reduces computational load through parameter-efficient fine-tuning and feature-level fusion.
Two variants, SpectralTrack and SpectralTrack+, show significant improvements over existing trackers.

Conclusions

SpectralTrack presents a novel and effective solution for hyperspectral video tracking.
The proposed methods enhance spectral-spatial feature utilization and temporal reasoning for improved tracking accuracy and efficiency.