A Novel Performance Evaluation Methodology for Single-Target Trackers.

Area of Science:

• Computer Vision
• Machine Learning
• Robotics

Background:

• Evaluating single-target tracker performance is crucial for advancing computer vision.
• Existing evaluation methods lack interpretability and robust comparison metrics.
• Standard datasets and systems may not fully capture real-world tracking challenges.

Purpose of the Study:

• To propose a novel, robust, and interpretable methodology for single-target tracker performance evaluation.
• To introduce a comprehensive, richly annotated dataset designed to maximize visual diversity.
• To present a flexible, multi-platform evaluation system for easy integration and comparison.

Main Methods:

• A ranking-based evaluation methodology considering statistical significance and practical differences.
• Development of a fully-annotated dataset with per-frame attributes, optimized for visual diversity via clustering.
• Implementation of a multi-platform evaluation system supporting third-party tracker integration.

Main Results:

• The proposed methodology, tested on the VOT2014 challenge with 38 state-of-the-art trackers, forms the largest benchmark to date.
• The new dataset proved highly challenging, with tested trackers outperforming standard baselines.
• An exhaustive analysis of tracking difficulty and a novel performance visualization technique were presented.

Conclusions:

• The developed evaluation methodology provides a simple and interpretable way to compare tracker performance.
• The sophisticated dataset and evaluation system facilitate more rigorous and reliable tracker benchmarking.
• This work significantly advances the field of single-target tracker evaluation and comparison.