Synthetic Data Generation for End-to-End Thermal Infrared Tracking

Area of Science:

• Computer Vision
• Machine Learning
• Deep Learning

Background:

• Deep networks excel in RGB visual tracking but struggle with thermal infrared (TIR) due to limited labeled data.
• Current TIR tracking methods often rely on handcrafted features, limiting performance.
• Bridging the data gap is crucial for advancing TIR visual tracking.

Purpose of the Study:

• To address the scarcity of labeled TIR datasets for training deep learning models.
• To leverage image-to-image translation for generating synthetic TIR data from RGB sources.
• To develop and evaluate end-to-end trained deep features for TIR visual tracking.

Main Methods:

• Utilized image-to-image translation (paired and unpaired) to convert RGB data into synthetic TIR data.
• Trained deep networks end-to-end on large-scale synthetic TIR datasets.
• Evaluated tracking performance on the VOT-TIR2017 benchmark dataset.
• Integrated motion features with the trained deep network.

Main Results:

• Networks trained on synthetic TIR data outperformed those trained on limited real TIR data.
• Combining synthetic and real TIR data further boosted performance.
• The proposed method, incorporating motion features, achieved over 10% relative improvement compared to state-of-the-art TIR trackers.
• Demonstrated the effectiveness of synthetic data for training optimal end-to-end TIR tracking features.

Conclusions:

• Image-to-image translation is a viable method for creating large labeled TIR datasets.
• End-to-end training on synthetic TIR data enables superior performance in TIR visual tracking.
• This approach significantly advances the state-of-the-art in thermal infrared visual tracking.