BihoT: A Large-Scale Dataset and Benchmark for Hyperspectral Camouflaged Object Tracking.

Area of Science:

• Computer Vision
• Remote Sensing
• Machine Learning

Background:

• Hyperspectral object tracking (HOT) is crucial for identifying camouflaged objects.
• Existing HOT datasets often exhibit bias, favoring visual over spectral features, limiting tracker performance in challenging scenarios.
• This bias hinders trackers from effectively utilizing spectral information when visual cues are unreliable.

Purpose of the Study:

• Introduce a new task: hyperspectral camouflaged object tracking (HCOT).
• Develop a large-scale dataset (BihoT) specifically for HCOT, featuring diverse camouflage scenes, similar appearances, varied spectrums, and frequent occlusion.
• Propose a novel baseline model, Spectral Prompt-based Distractor-Aware Network (SPDAN), to address the challenges of HCOT.

Main Methods:

• Constructed the BihoT dataset with 41,912 hyperspectral images (HSIs) across 49 video sequences.
• Developed SPDAN, comprising a Spectral Embedding Network (SEN) for spectral-spatial feature extraction, a Spectral Prompt-based Backbone Network (SPBN) for fine-tuning RGB trackers with spectral prompts, and a Distractor-Aware Module (DAM) for handling occlusion and distractors.
• SEN utilizes 3-D and 2-D convolutions; SPBN leverages spectral prompts to mitigate training sample scarcity; DAM employs a novel statistic and motion predictor to correct performance degradation.

Main Results:

• SPDAN demonstrated state-of-the-art performance on the BihoT dataset.
• The proposed model also achieved superior results on other existing HOT datasets.
• The effectiveness of SPDAN in hyperspectral camouflaged object tracking was validated through extensive experiments.

Conclusions:

• The new HCOT task and BihoT dataset provide a robust benchmark for evaluating trackers in challenging camouflage scenarios.
• SPDAN offers an effective solution for hyperspectral object tracking, particularly when objects are camouflaged and visual features are unreliable.
• The spectral prompt-based approach and distractor-aware mechanism significantly improve tracking accuracy in complex environments.